Data sample
Six tasks teleoperated on a Unitree G1 with Dex1-1 grippers: eggs, connect four, jenga, a puzzle, socks and wipes. Every recording is a LeRobot dataset, and carries far more than LeRobot asks for.
The recordings
Each preview shows the nine camera streams of one episode — the D435 on the body and a D405 on each wrist, in color and as the left and right infrared images of each stereo pair — above the LiDAR point cloud around the robot.
LeRobot compatible
An episode is a LeRobotDataset v3.0: the same meta/info.json, stats.json, tasks.parquet and episode index, one parquet of rows at 30 fps, and one AV1 video per camera. Anything that reads LeRobot reads it; the extra streams sit in their own folders beside it and stay out of its way.
The wipes episode, as it lands on disk:
meta/ info.json LeRobot features, with units for every column stats.json tasks.parquet episodes/chunk-000/file-000.parquet calibration.json intrinsics and extrinsics of every sensor depth.json lidar.json pico.json d435_imu.json data/chunk-000/file-000.parquet 2,921 rows, 97.4 s at 30 fps videos/ 9 AV1 camera streams observation.images.d435 848×480 color observation.images.d435_ir1, _ir2 848×480 infrared stereo observation.images.d405_left 424×240 color, left wrist observation.images.d405_left_ir1, _ir2 observation.images.d405_right 424×240 color, right wrist observation.images.d405_right_ir1, _ir2 depth/ 3 lossless 16-bit depth streams lidar/ Livox MID-360 points, packets and IMU pico/ raw teleoperation packets from the headset d435_imu/ D435i gyro and accelerometer
More than LeRobot
A LeRobot dataset holds a state, an action and some images. We record what a training team ends up asking for later, when it is too late to re-record:
- Lossless depth. On-chip depth from all three RealSense cameras as 16-bit FFV1, raw Z16 values with their depth scale — not a colormapped video.
- Stereo infrared. Both IR images of every camera, so depth can be recomputed with any stereo method.
- LiDAR. The Livox MID-360 on the torso: 12.7 million points in the wipes episode, the raw packets they came from and its built-in IMU, with no lost packets.
- Calibration. Intrinsics for every stream and the transform from every camera and the LiDAR into a robot link, each with its convention and where it came from.
- The whole control loop. Not just joint targets: the PD gains, feed-forward and velocity targets sent, the torque requested and commanded, the torque budget and every time it limited a joint, measured joint torque and motor temperature.
- Three IMUs. Pelvis and chest orientation, gyro and acceleration on every row, plus the D435i’s own IMU at 200 Hz gyro and 100 Hz accelerometer.
- The operator. The teleop command on every row — wrist poses, walking velocity, body height, buttons — and every raw packet from the PICO headset, received at up to 800 Hz, kept byte for byte beside the decoded fields.
- Timing you can check. Each row has its capture time, every camera’s hardware frame number and sensor timestamp, and its offset from the row; the LiDAR clock is mapped onto the same system clock, so every stream sits on one timeline.
If you need high-quality humanoid teleoperation datasets, contact us hello@rhoyn.com