Getting started

New Theory serves action policies for robots: you send what the robot is sensing, and a hosted model returns the poses it should move to next.

In this guide you install the SDK, authenticate with an API key, confirm the API answers, and — optionally — make a test inference call against a recorded observation. It takes about ten minutes, most of which is the install. The finish line is "I reached the API and it answered." No robot is required.

Already have newt installed and authenticated? Skip to Set up your embodiment to wire a real arm with a starter kit.

Three ways to use New Theory

New Theory exposes hosted models through three surfaces.

• API. A WebSocket endpoint that takes observation frames and returns action chunks. Use it when you write a custom client or call from a non-Python language. See the API reference.
• SDK. The newt Python library wraps the WebSocket, the msgpack framing, and the inference loop behind two callbacks and a robot.run(prompt) call. Most developers start here. See the SDK reference.
• Starter kits. Hardware-specific bundles that wire a real arm and cameras to the SDK. Use one when you move from recorded observations to a physical robot. See Set up your embodiment.

1. Install newt

Install newt as a global CLI tool from the public newt-python repo. The HTTPS command below clones anonymously — no GitHub account or access setup needed.

uv tool install "git+https://github.com/new-theory-research/newt-python.git"

On a machine set up for SSH-based GitHub auth instead, use the SSH URL:

uv tool install "git+ssh://git@github.com/new-theory-research/newt-python.git"

Requires Python 3.11 or newer.

Agent-driven install: git+ssh:// installs may require a human-approval step in your agent's harness — that's expected behavior, not a hang; approve and the install continues.

No uv yet? curl -LsSf https://astral.sh/uv/install.sh | sh, then source $HOME/.local/bin/env to put it on PATH — details in the full install guide.

This installs newt as a global command available in every shell — no environment to activate. If newt isn't on PATH after install, run uv tool update-shell to add ~/.local/bin.

2. Authenticate the SDK with a New Theory API key

newt login

Running newt login creates an API key and asks you to confirm you're pairing with this session. If you don't have an account yet, you'll need to make one. The command prints a URL and a pairing code: open the URL in a browser, confirm the code matches the one in your terminal, and approve. The CLI stores the key at ~/.nt/credentials, and the SDK and CLI read it from there automatically on subsequent runs — no shell export needed.

In environments without a browser (CI, an agent, a bare SSH session), set NT_API_KEY in the environment instead. The SDK and CLI both read it, and NT_API_KEY takes precedence over the stored credentials file when both are present.

To generate an API key manually, do so from the New Theory Console — see Key management for the full flow.

3. Check the model registry

newt models

Key nt_••••••••5bde16a3 (environment)

ft-64cad948c4e9f09c-red-cube-bowl  [base: ft_base_so101_ft]
ft-64cad948c4e9f09c-svla-so101-pickplace-20260714213146  [base: ft_base_so101_ft]
ft-64cad948c4e9f09c-my-first-pickplace  [base: ft_base_so101_ft]

molmoact2
    so101

The registry answers with a Key line naming which credential answered (masked — never the full key), then the available models grouped by family — base models with their action axes, fine-tunes beneath them. Raw uids are left out by default; pass --ids if you need them. This is your first contact with the API. If newt models returns a list, you're connected.

4. Connect from Python

The New Theory Python library installs into a project — you add it when you have Python code to write. Create one if you don't have it yet:

uv init my-robot
cd my-robot
uv add "newt @ git+https://github.com/new-theory-research/newt-python.git"

On a machine set up for SSH-based GitHub auth instead, use uv add "newt @ git+ssh://git@github.com/new-theory-research/newt-python.git".

Then confirm the API answers:

uv run python -c "from newt import Robot; print(Robot())"
# expected output:
# so101 · contract received · (30,6) · 6 labeled axes

Robot() resolves the default model and fetches its contract from the registry — output shape, axis labels, everything you need to know about what the model returns. It reads the same credentials newt login created — no second login, no environment variable needed.

You've successfully communicated with the API. The model contract is in your hands. Some developers are done at this step.

Make a test call

This step is optional and needs no hardware. Run this section if you want to see what the model returns before wiring anything physical.

This is a test call against a recorded observation. snapshots.load() replays a saved camera-and-state snapshot; the model returns what it would do given that snapshot. No robot is connected. Nothing moves anywhere. Run the code below in the project you set up in step 4, using uv run python.

Load a snapshot

from newt import Robot, snapshots

robot = Robot()
obs = snapshots.load("red_cube")

A snapshot is one recorded observation — robot state, camera frames, and the task prompt the episode was collected with. snapshots.available() lists all bundled recordings, including cup_stacking and pour_coffee_beans.

Run inference

response = robot.infer(obs)
print(response)
# action_chunk (30, 6): shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper | latency 261ms

The first inference call wakes the model's container — it can take a few minutes. Subsequent calls return in a few seconds.

That printed line is the model's response in one row: the action chunk shape, the labeled axes, and the round-trip latency. Nothing moved, because there's no robot in this call — you confirmed the request and response shapes a real integration builds on. For the full anatomy of the response — every field, what each axis means, latency behavior — see Calling the model.

If the call fails

• newt.AuthError — your key is wrong, revoked, or not visible in this shell. The registry validates the key during Robot() construction, so a bad key fails fast — before any connection opens — with a hint that valid keys start with nt_. Re-run newt login or reissue the key.
• A ColdStartRetry warning that lingers — the GPU container is warming up; first call after idle can take a few minutes. The call completes on its own, and later calls in the same session skip the wait.
• TimeoutError — the container did not become ready within the SDK's 180-second retry window. This is not a routine cold start. Retry once; if it persists, contact support.

Next: set up your embodiment

In the next guide you set up an embodiment: wire the SDK to a real arm and cameras and drive the model on hardware. Continue to Set up your embodiment — choose your hardware and follow its starter-kit walkthrough.