Series 5 tried reinforcement learning (RL) with a simple grade, and the model got worse: v5 learned to hover over the tube instead of picking it up. v6 keeps the idea and fixes both inputs that went wrong. Its corrections come from the improved expert rescue skill, and a finer grade counts hovering against a run.
What does this article cover?
What v6 changes from v5, how the v6 data was built, and how v6 did on the same 50-run test.
| Version | Training data | Loss | Success rate | Performance |
|---|---|---|---|---|
| v4 | v3 data + 25 DAgger corrections (Series 4) | 0.016 | 8 / 50 (16%) | Picked up the tube on its own, the first model to do so |
| v5 | v4 data + v4's own successes weighted by grade v1 + round 2 corrections (Series 5) | 0.022 | 5 / 50 (10%) | Hovered over the tube and rarely closed the gripper |
| v6 | v4 data + round 3: 43 continuous rescues + 5 v5 successes scored by grade v2 | 0.018 | 11 / 50 (22%) | The best so far. Quick, clean successes, but still often stops above the tube |
Success rate counts the runs where the model lifted the tube with no help.
How has each model done so far?
Share of test runs where the model picked up the tube with no help, across every trained model in the series. ACT is from Series 2, SmolVLA v1 to v3 from Series 3. The number under each bar is successes out of test runs.
What does v6 change from v5?
Series 5 traced v5's hovering to two inputs. The grade gave slow, hesitant successes the same weight as clean ones, and the round 2 corrections came from the old expert rescue skill, which lined up above the tube in stop-and-look steps. v6 replaces both.
| Input | v5 | v6 |
|---|---|---|
| Corrections (DAgger) | Round 2, from the old stop-and-look expert rescue skill | Round 3, from the improved expert rescue skill, one continuous motion from where the model stopped |
| Reward (RL) | Grade v1: lifted or not, faster than the median or not | Grade v2: a score that also counts hovering against a run |
| Base data | v4's data | v4's data, unchanged |
How does grade v2 score a run?
| Grade v1 (v5) | Grade v2 (v6) | |
|---|---|---|
| What it measures | Lifted or not, faster than the median or not | Score = 1 − 0.01 per second − 0.05 per second of hover |
| Copies of a success | 2, or 3 if faster than the median | 1 + 2 × score, so a fast clean run counts up to 3 times and a slow hovering one 1 or 2 |
| Hover in the kept frames | Only fully still frames trimmed | The hover phase cut to its last second before closing |
Hover is the seconds the jaws spend open within 3 cm of the table before they start closing.
How was the v6 dataset built?
v4 data + DAgger round 3 from v5's 50 test runs, scored by grade v2.
DAgger round 3: v5's 50 test runs
The 50-run test of v5 in Series 5 doubled as the third round of DAgger. v5 lifted the tube on its own 5 times. The improved expert rescue skill took over the other 45 runs and rescued 43, in one continuous motion from wherever v5 stopped.
What goes into v6
| Data | Into v6? | Why |
|---|---|---|
| Everything v4 trained on | Yes, unchanged | The demos and round 1 corrections got v4 to its first successes |
| v5's additions (round 2 corrections, v4 successes copied 2 to 3x) | No | The likely source of the hovering |
| Round 3 rescues (43 of v5's runs) | Expert rescue skill frames only | Continuous corrections from where the model actually stops. When a rescue needed a second try, only the second try is kept |
| Round 3 successes (5) | Yes, 1 to 3 copies by score | All five hovered 6 to 12 s, so they scored 0.23 to 0.58 and got 1 or 2 copies each |
| Failed rescues and edge cases | No | Not demonstrations |
How the data is prepared
- Collect. v5 drives 50 runs with the watcher on. Every frame is saved with both camera images, the joint positions and a tag for who was driving, the model or the expert rescue skill.
- Grade. Each run gets its outcome (success, empty grip, stall, timeout) and who finished it. Each success also gets its hover time and a score.
- Rescues. Only the expert rescue skill's frames are kept. The model's frames before the stall show what not to do. A missed first rescue try is cropped out using the logged start of the second try.
- Successes. Still stretches are trimmed, then the hover phase is cut to its last second before the jaws close. Trimming only still frames had kept 85% of the hover, because the model jitters rather than sitting still.
- Combine and check. v4's 72 episodes are copied unchanged and the round 3 episodes are added. Every episode is checked for matching frame counts across both cameras and the joints, and for missing or non-finite values.
The v6 dataset
| Episode type | Source | Runs | Episodes in training |
|---|---|---|---|
| Human teleop demo | Series 3 (v1) | 20 | 20 |
| Scripted demo | Series 3 (v2) | 27 | 27 |
| Fail + rescue, old skill | DAgger round 1 (v3) | 25 | 25 |
| Fail + rescue, continuous skill | DAgger round 3 (v5) | 43 | 43 |
| Model success, graded | DAgger round 3 (v5) | 5 | 9 |
| Total | 120 | 124 |
Runs are distinct recordings. Episodes in training counts the extra copies of graded successes. In frames: 48,473 in total, made of v4's 72 episodes, the 43 rescues' expert frames (13,722) and 5 scored successes as 9 copies. Left out: 2 failed rescues, v5's round 2 corrections and its copies of v4's successes.
How was model version 6 trained?
On the v6 dataset above, 20,000 steps from the SmolVLA base model with the same settings as v4 and v5, on a Runpod H100 at $3.99/h. That is about 0.58 s per step and $14.28 in total. Final loss 0.018, between v4's 0.016 and v5's 0.022.
How did v6 do over 50 runs?
Same 50-run test as v4 and v5. v6 alone: 11 of 50 (22%), the best of any model so far.
v6 picks up the tube on its own, about 1.7× speed (5 s). It reaches, closes and lifts in 9 seconds with no hovering.
Results
| Measure | v4 | v5 | v6 |
|---|---|---|---|
| Model picked up the tube alone | 8 / 50 (16%) | 5 / 50 (10%) | 11 / 50 (22%) |
| Expert rescue skill rescued the rest | 33 / 42 (79%) | 43 / 45 (96%) | 39 / 39 (100%) |
All three tests used the same rule: the watcher calls a stall after 4 seconds without movement. The v6 test ran in daylight and the v4 test in the evening, so lighting may account for part of the gap.
What it says about DAgger and RL
Better rescue data gave a better model. v6 is up from v5's 10% and v4's 16%, and its successes are quick, 9 to 16 seconds, so the hover penalty did its job on the runs that work.
The gain is modest. Most failures still look the same: v6 reaches the tube in 4 to 9 seconds, stops with the jaws open just above or behind it, and stays frozen until the watcher hands over. In 32 of the 33 stalls the model never sent a close command. Cleaner gripping in the rescues has not yet taught the model to commit to the grip.
What did watching v6 show about the grip?
Watching v6 run showed two problems with the grip itself, both copied from the training data.
- A grip that is too loose. The expert rescue skill and the scripted demos close until the jaws touch the tube, then squeeze only about 1.5 more. When the model does close, it copies that light hold, and with a little noise the tube slips out. Too firm is a problem too: a full close can flip the tube out of the jaws.
- A grip too far from the cap. The expert rescue skill aimed at the exact middle of the tube. Toward the crimped end the tube is flat and harder to hold than the round body nearer the cap.
The expert rescue skill now squeezes 4 past contact, firm enough to hold but short of a full close, and aims 12% of the tube's length toward the cap from the middle.