# Calling the model
URL: /docs/calling-the-model

The robot.infer() call and the InferenceResponse it returns — action chunk, axis labels, latency.



`robot.infer(obs)` sends one observation to the model and returns an `InferenceResponse` — the action chunk the model predicts for that observation, plus the labels and timing you need to interpret it. For a walkthrough of making this call against a recorded snapshot, see [Getting started](/docs/getting-started#make-a-test-call).

## The call [#the-call]

```python
from newt import Robot, snapshots

robot = Robot()
obs = snapshots.load("red_cube")
response = robot.infer(obs)
```

`infer()` takes one observation — robot state plus camera frames in the model's contract shape. A snapshot from `snapshots.load()` satisfies the contract out of the box; a real integration passes reads from its own hardware. One observation in, one response back; no loop runs and no robot is involved.

## `InferenceResponse` [#inferenceresponse]

```python
print(response)
# action_chunk (30, 6): shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper | latency 261ms
```

The printed form shows the three fields in one row: chunk shape, axis labels, round-trip latency.

| Field          | What it is                                                                                                                                                                                                                                |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `action_chunk` | `(30, 6)` float32 ndarray — 30 target joint configurations, one per row.                                                                                                                                                                  |
| `axes`         | `['shoulder_pan', 'shoulder_lift', 'elbow_flex', 'wrist_flex', 'wrist_roll', 'gripper']` — semantic label for each action dimension.                                                                                                      |
| `latency_ms`   | Round-trip time for the final attempt's send and receive, in milliseconds. Excludes connect time and any cold-start retry.                                                                                                                |
| `model`        | Resolved model identifier (UID or tag), or `None` when it isn't known.                                                                                                                                                                    |
| `total_ms`     | Wall-clock time for the whole `infer()` call, in milliseconds — connect (including any cold-start retry), every failed attempt and its backoff, and the final send/recv. Equals `latency_ms` when nothing but the final attempt happened. |
| `retries`      | Count of retries the call needed — a cold-start connect retry and/or transient verifier retries. `0` on the common warm path.                                                                                                             |

## Action chunk [#action-chunk]

Each row of the chunk is one target joint configuration: `shoulder_pan`, `shoulder_lift`, `elbow_flex`, `wrist_flex`, `wrist_roll` in radians, and a normalized `gripper` value. Rows are ordered in time — the model's plan for the next 30 steps.

## Axis labels [#axis-labels]

`axes` comes from the model's registry contract (`action_axes`), so you read what each dimension means instead of guessing from float indices. The labels arrive with the contract at `Robot()` construction — see [model discovery](/docs/api#model-discovery) in the API reference.

## Latency [#latency]

`latency_ms` measures one round trip. The first call after an idle period wakes the model's container; that cold start can take a few minutes. If the first connect attempt times out, the SDK retries once and surfaces a `ColdStartRetry` warning while it waits. `total_ms`, not `latency_ms`, is the field that includes this cold-start time — check it when a call takes longer than the round-trip figure suggests. Warm calls return in a few seconds.

## Other models [#other-models]

The chunk shape and axis labels vary by model. [MA2-SO-101](/docs/models/molmoact2/so101) returns `(30, 6)` joint configurations. The registry contract is the source of truth for any model's shapes — see [model discovery](/docs/api#model-discovery) and the per-model pages under [Models](/docs/models).