# SO-101
URL: /docs/models/molmoact2/so101
Fine-tune of MolmoAct-2 for the SO-101 single-arm embodiment. 6D joint state, two cameras, 30-step action chunks. UID ft_6341c5_d13da9.
fine-tune
so101
`ft_6341c5_d13da9` — MolmoAct-2 fine-tuned on the SO-101 single-arm embodiment. 6D joint state, two RGB cameras in any order, 30-step action chunks. Norm tag `so100_so101_molmoact2`. \[source: [https://huggingface.co/allenai/MolmoAct2-SO100\_101](https://huggingface.co/allenai/MolmoAct2-SO100_101)]
## Quickstart [#quickstart]
```python
import os
import newt
import numpy as np
def read_state():
return {
"state": np.zeros(6, dtype=np.float32),
"images": {
"top": np.zeros((3, 224, 224), dtype=np.uint8),
"side": np.zeros((3, 224, 224), dtype=np.uint8),
},
}
def execute(chunk):
for joints in chunk:
pass # send 6D joint targets to your controller
robot = newt.Robot(
api_key=os.environ["NT_API_KEY"],
model="so101",
read_state=read_state,
execute=execute,
)
result = robot.run("place the block in the bin", max_duration=30.0)
```
Replace `np.zeros(...)` with your actual sensor reads. `state` is 6 floats (single-arm joint positions); each image is channels-first uint8 at 224×224. Neither camera key is individually required — a missing key zero-fills with a `DegradationWarning`, and camera order is not fixed. The [SO-101 starter](https://github.com/new-theory-research/newt-starter-so101) ships both callbacks in `embodiment.py` already wired to real lerobot `SO101Follower` hardware — what you fill in is `nt.toml` (serial port, camera indices), not the callback bodies.
## Inputs [#inputs]
`read_state` must return a dict with these keys:
| Key | Shape | Dtype | Notes |
| ------------- | --------------- | --------- | --------------------------------------------------------------------------- |
| `state` | `(6,)` | `float32` | 6D single-arm joint state. |
| `images.top` | `(3, 224, 224)` | `uint8` | Top-view camera, CHW, RGB. Missing key zero-fills with DegradationWarning. |
| `images.side` | `(3, 224, 224)` | `uint8` | Side-view camera, CHW, RGB. Missing key zero-fills with DegradationWarning. |
The server validates the `state` shape and image dimensions on the first observation frame. A `state` shape mismatch closes with code `4422`. Camera mismatches are permissive — neither key is hard-required; see [Camera roles](#camera-roles) below. See the [API reference](/docs/api) for error envelope details.
## Outputs [#outputs]
`execute` receives a chunk of shape `(30, 6)` — a 30-step action horizon, 6 action dims per step. Each row is a target joint configuration in the same 6D format as `state`.
The 6 action dims correspond to `["shoulder_pan", "shoulder_lift", "elbow_flex", "wrist_flex", "wrist_roll", "gripper"]` in that order (see [State layout](#state-layout)).
Clients typically execute a prefix of the chunk, then read fresh state and send the next observation. See the [API reference](/docs/api#what-you-get-back) for the receding-horizon pattern.
## Provenance [#provenance]
| Field | Value |
| ------------------- | -------------------------------- |
| Checkpoint UID | `ft_6341c5_d13da9` |
| Base | MolmoAct-2 (`ft_base_molmoact2`) |
| Upstream checkpoint | `allenai/MolmoAct2-SO100_101` |
| Norm tag | `so100_so101_molmoact2` |
| Precision | bf16 |
## State layout [#state-layout]
The 6D `state` vector maps to the SO-101's joint axes, gripper-last:
| Index | Name | Units |
| ----- | --------------- | ---------- |
| 0 | `shoulder_pan` | Radians |
| 1 | `shoulder_lift` | Radians |
| 2 | `elbow_flex` | Radians |
| 3 | `wrist_flex` | Radians |
| 4 | `wrist_roll` | Radians |
| 5 | `gripper` | Normalized |
Send these in the same layout to `read_state`. The SO-101 is not factory-calibrated — run lerobot's calibration flow before the first run.
## Camera roles [#camera-roles]
The two expected cameras serve fixed viewing angles in the training distribution:
| Camera key | Mount | What it sees |
| ---------- | ---------------- | ----------------------- |
| `top` | Overhead | Workspace from above |
| `side` | Fixed side mount | Workspace from the side |
Camera order is not fixed; the server does not consume them positionally. A missing key zero-fills with a `DegradationWarning` rather than closing with `4422`.
## Notes [#notes]
* **`max_action_horizon` (30) vs `flow_matching_num_steps` (10)** — `max_action_horizon` is the action chunk length you receive (30 steps); `flow_matching_num_steps` is the internal diffusion solver step count (10). HuggingFace inference examples pass `num_steps=10` referring to the latter; that value does not describe the action chunk shape.
* **Gripper axis (axis 6):** negative-signed on synthetic observations. Physical open/close semantics are unverified without hardware — the sign convention may need to be flipped on a real SO-101. Verify against your arm before deploying.
* **Eval-parity:** task-success rate against the Allen AI baseline has not been benchmarked with physical hardware. A success-rate gap should be treated as a potential ingestion issue rather than a model quality ceiling until verified on a real arm.
* To add a customer fine-tune on top of this checkpoint, declare a registry entry with `base: ft_6341c5_d13da9` — the contract is inherited automatically via recursive resolution.
## References [#references]
* Model card: [allenai/MolmoAct2-SO100\_101](https://huggingface.co/allenai/MolmoAct2-SO100_101)
* Paper: [arXiv:2605.02881](https://arxiv.org/abs/2605.02881)
* Upstream code: [github.com/allenai/molmoact2](https://github.com/allenai/molmoact2)