New Theory SDK

The New Theory SDK is a Python library for the New Theory inference API. You write two callbacks — one that returns what your robot is sensing, one that moves your robot — and call robot.run(prompt). The SDK handles the network connection, the message format, and the inference loop.

Source repo: new-theory-research/newt-python. Import name: newt.

What it does

• The loop. Calls your read_state() to get an observation, sends it to the model, receives the plan, calls your execute(chunk) with it. Repeats until the model emits a terminal frame or max_duration elapses.
• Connection. Opens one authenticated WebSocket per robot.run() call, holds it for the duration of the task, closes it cleanly when the run ends.
• Framing. Encodes observation dicts as msgpack (with the msgpack-numpy ndarray extension) on the way out; decodes action chunks back into NumPy arrays on the way in. You never touch the wire format.
• Bootstrap. On Robot() construction, the SDK contacts the always-on registry to resolve the model tag and fetch its contract. A bad or revoked key fails here, before any WebSocket is opened, with AuthError and a hint that valid keys start with nt_. If the registry itself is unreachable, RegistryUnavailable is raised (check NT_BOOTSTRAP_URL or retry). Two environment overrides exist for infra testing: NT_BOOTSTRAP_URL replaces the registry URL and NT_INFERENCE_URL pins the inference endpoint. Leave both unset for production runs.
• Auth. Resolves your API key — NT_API_KEY env var first, then ~/.nt/credentials — and presents it as a Bearer token on the WebSocket handshake. A key revoked mid-session raises AuthError at the next connection.
• Error mapping. Translates WebSocket close codes into typed Python exceptions, all subclasses of newt.NewTheoryError: AuthError (4001), ForbiddenError (4403), ProtocolError (4400), ModelNotFoundError (4404), ContractMismatchError (4422), ServerError (4500), and VerifierError (4503) — plus a codeless ConnectionDroppedError for abnormal drops. The full close-code table lives in the API reference.

What it doesn't do

The SDK does not run inference — the model lives on New Theory's hosted endpoint, and the SDK is the client that calls it. It does not drive the robot; the SDK hands action chunks to your execute callback, which decides how to apply them. It does not bundle a simulator, a viewer, or a training stack.

Call shape

import os
import newt

robot = newt.Robot(
    api_key=os.environ["NT_API_KEY"],
    read_state=read_my_robot,    # you write this
    execute=move_my_robot,       # you write this too
    model="so101",
)

robot.run("pick up the cup", max_duration=30.0)

newt.Robot(...) takes your API key, two callbacks, and the model name. robot.run(prompt, max_duration=...) opens the connection, drives the loop, and returns when the model signals the task is complete or max_duration elapses. The return value names how the run ended.

See Set up your embodiment for wiring the callbacks into real hardware.

Embodiment

An embodiment is any object that drives your hardware. It implements two methods, and you pass it to Robot(embodiment=obj) instead of the two callbacks.

class Embodiment(Protocol):
    def read_state(self) -> dict: ...
    def execute(self, action_chunk: np.ndarray) -> None: ...

read_state() returns one observation dict — the same shape read_state= returns. execute(action_chunk) applies one action chunk, an ndarray of shape (action_horizon, action_dim). The protocol is runtime_checkable, so any object with both methods qualifies; no inheritance or registration is required.

Robot(embodiment=obj) is equivalent to passing read_state=obj.read_state and execute=obj.execute separately. The two paths produce the same run. Passing both embodiment= and either callback raises EmbodimentError with type embodiment.conflict — they are mutually exclusive. Passing a string raises EmbodimentError with type embodiment.string_not_object, because the SDK consumes objects and never resolves names. See Embodiment errors for all three failure types.

Bare callbacks remain supported and undeprecated. embodiment= is shorthand for the object you already build; if your style is two closures, two closures still work.

Construction

Starter kits stamp an embodiment class into your repo as embodiment.py. You construct it and pass the instance:

from newt import Robot
from embodiment import SO101

robot = Robot(embodiment=SO101.from_config())
robot.run("Pick up the red cup")

from embodiment import SO101 resolves because the file lives in your repo — the starter kit put it there. from_config() reads the hardware addresses from ~/.config/nt/nt.toml. The class is yours to rename and edit. To write your own for any hardware, see Set up your embodiment.

Install

The newt SDK lives in the public new-theory-research/newt-python repo. The HTTPS command clones anonymously — no GitHub account or access setup needed:

pip install git+https://github.com/new-theory-research/newt-python.git

On a machine set up for SSH-based GitHub auth instead, use the SSH URL:

pip install git+ssh://git@github.com/new-theory-research/newt-python.git

Then import newt works.

Try it

For the full walk-through — issuing a key, running the script, reading what comes back — see Getting started.