OpenArm: D435F, on the column, tilted 20°
- Color and IR
- AV1, 848×480, 30 Hz. Both infrared images, so depth can be recomputed with any stereo method.
- Depth
- Lossless 16-bit FFV1, 424×240, 15 fps. Raw Z16 values with their depth scale.
The depth colors in the preview are only for viewing.
OpenArm: D405, on each wrist
- Color and IR
- AV1, 424×240, 30 Hz
- Depth
- Same as the column camera
Depth value 0 means no stereo match, and 65535 saturated or out of range. This happens on any camera, but mostly on these short-range D405s.
OpenArm: joints and motors
- Action
- 16 joint targets: 14 arm joints and 2 grippers, all following the operator
- Joint state
- Position, velocity, measured torque and motor temperature of 14 arm joints; aperture and torque of 2 grippers
- Motor commands
- PD gains, feed-forward torque and velocity targets; torque requested, capped and commanded
- Torque limits
- A budget for every arm joint and gripper, and a flag each time it held one back
OpenArm: PICO 4 Ultra, on our own teleop stack
- On every row
- Wrist poses, triggers, grips, sticks and buttons
- Raw packets
- Every headset packet, received at about 90 Hz, kept byte for byte
OpenArm: sync and calibration
- One clock
- A capture time on every row, on one system clock
- On every row
- Each camera’s hardware frame number, sensor timestamp and offset from the row
- Dropped data
- Frame numbers show every dropped camera frame; each episode counts corrupt headset packets
- Calibration
- Intrinsics for every stream, and the transform from every camera into a robot link
Cameras run free, not hardware-triggered: frames usually land within about ±16.7 ms of their row. Around dropped frames, about 2–5% of rows land further off, up to about 117 ms. Every row records the exact offset.
Extrinsics come from the robot model, not a hand-eye calibration. Refine them for precise depth-to-robot alignment.
Depth runs at half the row rate: depth frame i belongs to row 2i.
reBot: D435F, on a post, tilted 20°
- Color and IR
- AV1, 848×480, 30 Hz. Both infrared images, so depth can be recomputed with any stereo method.
- Depth
- Lossless 16-bit FFV1, 424×240, 15 fps. Raw Z16 values with their depth scale.
The depth colors in the preview are only for viewing.
reBot: D405, on each wrist
- Color and IR
- AV1, 424×240, 30 Hz
- Depth
- Lossless 16-bit FFV1, 424×240, 15 fps
Depth value 0 means no stereo match, and 65535 saturated or out of range. This happens on any camera, but mostly on these short-range D405s.
reBot: joints and motors
- Action
- 14 joint targets: 12 arm joints and 2 grippers, all following the operator
- Joint state
- Position, velocity, measured torque and motor temperature of 12 arm joints; aperture and torque of 2 grippers
- Motor commands
- PD gains, feed-forward torque and velocity targets; torque requested, capped and commanded
- Torque limits
- A budget for every arm joint and gripper, and a flag each time it held one back
reBot: PICO headset, on our own teleop stack
- On every row
- Wrist poses, triggers, grips, sticks and buttons
- Raw packets
- Every headset packet, received at about 90 Hz, kept byte for byte
reBot: sync and calibration
- One clock
- A capture time on every row, on one system clock
- On every row
- Each camera’s hardware frame number, sensor timestamp and offset from the row
- Dropped data
- Frame numbers show every dropped camera frame; each episode counts corrupt headset packets
- Calibration
- Intrinsics for every stream, and the transform from every camera into a robot link
Cameras run free, not hardware-triggered: frames usually land within about ±16.7 ms of their row. Around dropped frames, about 5–7% of rows land further off, up to about 92 ms. Every row records the exact offset.
Extrinsics come from the robot model, not a hand-eye calibration. Refine them for precise depth-to-robot alignment.
Depth runs at half the row rate: depth frame i belongs to row 2i.
Unitree G1: D435i, on the head
- Color and IR
- AV1, 848×480, 30 Hz. Both infrared images, so depth can be recomputed with any stereo method.
- Depth
- Lossless 16-bit FFV1, 424×240, 15 fps. Raw Z16 values with their depth scale.
Unitree G1: D405, on each wrist
- Color and IR
- AV1, 424×240, 30 Hz
- Depth
- Same as the head camera
Depth value 0 means no stereo match, and 65535 saturated or out of range. This happens on any camera, but mostly on these short-range D405s.
Unitree G1: Livox MID-360, in the head
- Recorded
- Points, the raw packets they came from and its built-in IMU
- Wipes episode
- 12.7 million points, no lost packets
Unitree G1: three IMUs
- Pelvis and chest
- Orientation, gyro and acceleration on every row
- D435i
- 200 Hz gyro, 100 Hz accelerometer
Unitree G1: joints and motors
- Action
- 31 joint targets: 14 arm joints and 2 grippers follow the operator; 15 leg and waist joints come from a learned locomotion policy the operator steers with the sticks
- Joint state
- Position, velocity, measured torque and motor temperature of 29 joints; aperture and torque of 2 grippers
- Motor commands
- PD gains, feed-forward torque and velocity targets; torque requested, capped and commanded
- Torque limits
- A budget for every arm joint and gripper, and a flag each time it held one back
Unitree G1: PICO 4 Ultra, on our own teleop stack
- On every row
- Wrist poses, triggers, grips, sticks, buttons, walking velocity and body height
- Raw packets
- Every headset packet, received at about 90 Hz, kept byte for byte
- Hand latency
- 20 ms
Unitree G1: sync and calibration
- One clock
- A capture time on every row, on one system clock, with the LiDAR clock mapped onto it
- On every row
- Each camera’s hardware frame number, sensor timestamp and offset from the row
- Dropped data
- Frame numbers show every dropped camera frame; each episode counts lost LiDAR packets and corrupt headset packets
- Calibration
- Intrinsics for every stream, and the transform from every camera and the LiDAR into a robot link
Cameras run free, not hardware-triggered: frames land about ±16.7 ms from their row, and up to about 30 ms apart across cameras.
Extrinsics come from the robot model, not a hand-eye calibration. Refine them for precise depth-to-robot alignment.
Depth runs at half the row rate: depth frame i belongs to row 2i.