Walk slowly and precisely
When you drive a humanoid by hand, most of what you ask for is small: ten centimetres closer to the counter, a tenth of a radian to face the shelf. We tested how thirty-four open-source walking policies for the Unitree G1 handle those small requests. Most ignore them. A few seem to obey but are really just drifting.
github.com/rhoyn/teleop-walking-benchmark
The policies are other people’s work; we only measure them. Everything below is in simulation, MuJoCo and PhysX, except the robot test.
Drive it yourself. This is gr00t_wbc at 0.74 m, the best policy below, running in your browser on MuJoCo. Click the robot, then W/S walks forward and back, A/D steps sideways and Q/E turns. Pick how big each request is:
You asked for 0.2 rad/s. The robot gave you zero.
We tried it on our own G1. We asked it to turn on the spot, raising the request slowly from nothing to 0.8 rad/s. It turned at roughly a fifth of what we asked, in fits and starts.
Then we held three fixed speeds. Asked for 0.2 rad/s, it did not turn at all. Asked for 0.4, it crept round at 0.01 rad/s. Asked for 0.6, it managed 0.1 — a sixth of the request. It never really stepped; it shuffled, lifting its feet less than a centimetre.
The policy was gr00t_wbc, crouched at 0.66 m — the height that won our punch benchmark.
Why policies ignore small requests
Mostly because they were built to. Small requests get thrown away in three places.
- In training. The common training code, legged_gym and Unitree’s unitree_rl_gym, replaces any walking request under 0.2 m/s with zero, so the robot never practises walking slowly. HOMIE also rewards the robot for standing still.
- In the authors’ own control code. OpenWBT ignores anything under 0.1. GR00T hands over to a separate standing network under 0.05. AMO stands still below 0.1 m/s and cannot turn on the spot at all.
- In the motors. A policy trained with different gravity, body mass or motor stiffness and damping (KP/KD) than the real robot expects its joints to hold firmer than they do. On the robot the motors give way under the body’s weight, and the feet never leave the floor. The policy is trying to step; the robot keeps both feet on the ground and drags them. What looks like shuffling is steps the motors failed to lift: on our G1 the policy asked the ankle to swing half a radian and it moved 0.03.
REFINE-DP describes the same problem and suggests placing footsteps instead. Nobody had measured it across policies, so we did.
The test
Each run gives the robot one request and holds it for ten seconds: turn at a set speed, or walk at a set speed in a set direction — forwards, backwards or sideways. Requests go up to 1 m/s or 1 rad/s. Every policy ran 4096 times per request type, per simulator. We measure what the robot actually did from the second second to the end, and drop runs where it fell.
Our first scoring looked only at speed, and it was misleading: a robot that wanders off at 13 cm/s no matter what you ask looked like the most responsive one. So we score small requests, 0.05 to 0.15, by how far the robot moved in the direction it was asked.
In this video every robot was asked to turn right on the spot at 0.1 rad/s and nothing else — stand in place and rotate. Any walking forward, backward or sideways you see is drift, and it counts as error.
In this one every robot was asked to walk straight forward at 0.1 m/s and nothing else — no turning, no sidestep. Any turning or sliding sideways you see is drift, and it counts as error.
Results
Twenty-two of the forty-two configurations stay upright reliably enough to score. The columns:
- Error — how far off the robot was, as a share of what it was asked. 0% is perfect. 100% is what you get by standing still. Over 100% means it went the wrong way.
- Right way — how often it moved in the direction asked.
- Delivered — how much of the requested speed it reached.
- Drift at zero — how fast it moves when asked to stand still, in MuJoCo and PhysX.
| policy | legs KP/KD | error | walk: right way | walk: delivered | drift at zero m/s | turn: right way | turn: delivered |
|---|---|---|---|---|---|---|---|
| gr00t_wbc_h074_p000 | x1.0 | 37% | 97% | 0.85 | 0.001 / 0.002 | 99% | 0.90 |
| openwbt | x1.0 | 62% | 90% | 0.79 | 0.042 / 0.053 | 99% | 0.75 |
| gr00t_wbc_h070_p000 | x1.0 | 65% | 80% | 0.58 | 0.001 / 0.002 | 86% | 0.59 |
| gr00t_wbc_h066_p012 | x1.5 | 69% | 85% | 0.53 | 0.001 / 0.001 | 83% | 0.60 |
| wbc_agile_velocity | x1.0 | 75% | 11% | 0.02 | 0.002 / 0.001 | 69% | 0.54 |
| gr00t_wbc_h066_p000 | x1.5 | 77% | 77% | 0.53 | 0.001 / 0.001 | 81% | 0.59 |
| zealot | x1.0 | 85% | 50% | 0.36 | 0.011 / 0.001 | 21% | 0.09 |
| wbc_agile | x1.0 | 88% | 37% | 0.11 | 0.003 / 0.003 | 16% | 0.17 |
| sonic | x1.0 | 93% | 6% | 0.01 | 0.006 / 0.007 | 44% | 0.22 |
| mimic_lite | x1.0 | 96% | 10% | 0.01 | 0.009 / 0.005 | 49% | 0.26 |
| clobot_with_arms | x1.0 | 97% | 17% | 0.00 | 0.001 / 0.001 | 28% | 0.07 |
| decoupled_wbc_h070_p000 | x1.0 | 99% | 26% | 0.01 | 0.001 / 0.010 | 24% | 0.03 |
| handoff_with_arms | x1.0 | 99% | 48% | 0.01 | 0.000 / 0.000 | 51% | 0.52 |
| decoupled_wbc_h074_p000 | x1.0 | 100% | 11% | 0.00 | 0.001 / 0.001 | 1% | 0.01 |
| homie | x1.0 | 100% | 14% | 0.00 | 0.001 / 0.001 | 0% | 0.01 |
| rl_lab | x1.0 | 100% | 24% | 0.01 | 0.001 / 0.002 | 0% | 0.01 |
| decoupled_wbc_h066_p012 | x1.0 | 100% | 29% | 0.00 | 0.000 / 0.124 | 25% | 0.02 |
| amo | x1.0 | 100% | 44% | 0.05 | 0.002 / 0.005 | 77% | 1.07 |
| robomimic | x1.0 | 101% | 13% | 0.00 | 0.004 / 0.004 | 20% | 0.08 |
| decoupled_wbc_h066_p000 | x1.0 | 101% | 32% | 0.00 | 0.001 / 0.153 | 27% | 0.01 |
| falcon | x1.0 | 135% | 69% | 0.82 | 0.132 / 0.112 | 100% | 0.95 |
| asap | x1.0 | 172% | 69% | 0.75 | 0.129 / 0.117 | 66% | 0.52 |
gr00t_wbc at 0.74 m is the clear winner. It goes the right way almost every time, delivers 85–90% of what you ask, and stands perfectly still when you let go. 0.74 m is the height its authors ship. The lower, crouched heights survive punches better but lose this precision.
The same chart for every policy — how it turns and how it walks for each request, in both simulators and at every leg stiffness — is in results/cmd_vs_real_movements.
openwbt follows small requests but creeps forward at 4–5 cm/s whenever it is walking, even while turning on the spot. It only stops creeping when the request is exactly zero. OpenWBT’s own cut-off at 0.1 hides this. It also falls much sooner under punches: 34 seconds on average, against over 80 for gr00t_wbc.
falcon and asap only look responsive. Asked to stand still, they wander at 11–14 cm/s, and a third of the time they go the wrong way.
The rest barely move. homie, rl_lab and decoupled_wbc at 0.74 m will not turn until asked for 0.4–0.6 rad/s. decoupled_wbc at 0.66 m stands still in MuJoCo but drifts at 12–15 cm/s in PhysX — same network, different simulator.
What works and what does not
- Works: cancel openwbt’s creep. Subtracting 4.5 cm/s from its forward speed whenever it walks halves its error on small requests, with no extra falls. The right amount differs between simulators, so on the robot it has to be measured.
- Works: fix homie’s turn scaling. homie’s robot code shrinks turn requests by four; its training code shrinks them by two, next to a note saying
TODO change this. With the training value, homie starts turning at 0.3 rad/s instead of 0.6, and then turns at almost exactly the speed asked. - Explains it, but no fix: stiffer legs. In simulation the policies ask for 1.5–5.5 cm of foot lift and get under a centimetre, because the standing knee sags. Making the leg motors 1.5× stiffer lets gr00t_wbc at 0.66 m turn from 0.07 rad/s instead of 0.4. But it makes almost every policy fall more under punches — openwbt lasts 85% less — and burns far more energy. Each policy was trained for its own stiffness.
- Does not work: rescaling the request. Multiplying a request cannot turn zero into movement. Adding a constant overshoots big requests and turns every tiny correction into a real step.
Why we wrote this
To help you pick the right policy for your job. Every one of them has strengths and weaknesses: gr00t_wbc at 0.74 m places the robot precisely but falls sooner under punches; the crouched heights survive punches but ignore small requests; openwbt follows every request but creeps. Most of the time the top policies are interchangeable, depending on what you need the robot to do — but only if you know where each one falls short.
And to get small, precise movements taken seriously. Today few policies support them well. We hope that showing the gap will bring more policies that do it right.
Everything is in github.com/rhoyn/teleop-walking-benchmark.
If you need high-quality humanoid teleoperation datasets, contact us hello@rhoyn.com