Calling the model
The robot.infer() call and the InferenceResponse it returns — action chunk, axis labels, latency.
robot.infer(obs) sends one observation to the model and returns an InferenceResponse — the action chunk the model predicts for that observation, plus the labels and timing you need to interpret it. For a walkthrough of making this call against a recorded snapshot, see Getting started.
The call
from newt import Robot, snapshots
robot = Robot()
obs = snapshots.load("red_cube")
response = robot.infer(obs)infer() takes one observation — robot state plus camera frames in the model's contract shape. A snapshot from snapshots.load() satisfies the contract out of the box; a real integration passes reads from its own hardware. One observation in, one response back; no loop runs and no robot is involved.
InferenceResponse
print(response)
# action_chunk (30, 6): shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper | latency 261msThe printed form shows the three fields in one row: chunk shape, axis labels, round-trip latency.
| Field | What it is |
|---|---|
action_chunk | (30, 6) float32 ndarray — 30 target joint configurations, one per row. |
axes | ['shoulder_pan', 'shoulder_lift', 'elbow_flex', 'wrist_flex', 'wrist_roll', 'gripper'] — semantic label for each action dimension. |
latency_ms | Round-trip time for the final attempt's send and receive, in milliseconds. Excludes connect time and any cold-start retry. |
model | Resolved model identifier (UID or tag), or None when it isn't known. |
total_ms | Wall-clock time for the whole infer() call, in milliseconds — connect (including any cold-start retry), every failed attempt and its backoff, and the final send/recv. Equals latency_ms when nothing but the final attempt happened. |
retries | Count of retries the call needed — a cold-start connect retry and/or transient verifier retries. 0 on the common warm path. |
Action chunk
Each row of the chunk is one target joint configuration: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll in radians, and a normalized gripper value. Rows are ordered in time — the model's plan for the next 30 steps.
Axis labels
axes comes from the model's registry contract (action_axes), so you read what each dimension means instead of guessing from float indices. The labels arrive with the contract at Robot() construction — see model discovery in the API reference.
Latency
latency_ms measures one round trip. The first call after an idle period wakes the model's container; that cold start can take a few minutes. If the first connect attempt times out, the SDK retries once and surfaces a ColdStartRetry warning while it waits. total_ms, not latency_ms, is the field that includes this cold-start time — check it when a call takes longer than the round-trip figure suggests. Warm calls return in a few seconds.
Other models
The chunk shape and axis labels vary by model. MA2-SO-101 returns (30, 6) joint configurations. The registry contract is the source of truth for any model's shapes — see model discovery and the per-model pages under Models.