Demos taught SmolVLA where to go, but never how to finish from where it actually ends up. In the previous post three versions trained on demos never picked up the tube. So this time the model drives, a watcher spots when it gets stuck, an expert rescue skill takes over, and the corrections become the next training data.
What does this article cover?
Two versions: v3 from the previous post running with a watcher, and v4 trained on the corrections it collected. Then, in an appendix, how the expert rescue skill was improved, using a third camera to see what went wrong.
| Version | Training data | Loss | Success rate | Performance | Root cause | So the next dataset |
|---|---|---|---|---|---|---|
| v3 | 47 demos, pauses trimmed (previous post) | 0.014 | 0 / 30 (0%) | Closed the gripper, but the jaws shut beside the tube | Demos show the ideal path, never how to recover from where the model actually ends up | let v3 drive and recorded the expert rescue skill's 25 corrections (DAgger round 1) |
| v4 | v3 data + 25 DAgger corrections | 0.016 | 8 / 50 (16%) | Picked up the tube on its own for the first time | Still weak near the base and sometimes closes on nothing. Its own successful runs are not in the data yet | add RL: grade every run and train on the model's own successes, weighted by grade (next post) |
Success rate counts the runs where the model lifted the tube with no help.
What is DAgger?
DAgger, short for Dataset Aggregation (Ross, Gordon and Bagnell, CMU, 2011), is the standard fix for a model that gets lost in places the demos never went. Let the model drive, step in when it goes wrong, and add the correction to the training data. The model then learns how to finish from its own mistakes, not just from a perfect demo.
Usually a person does the stepping in. They watch every run and, when the model goes wrong, take over with a leader arm or another teleoperation setup. Their moves become the corrections. That works, but it needs a person at the robot for every run, and that doesn't scale past a few robots.
What are the watcher and the expert rescue skill?
In this series, two pieces of software share the person's job, and a person is only needed for edge cases. They are new terms used throughout the rest of this article.
| Watcher | The judge. It never moves the arm. It checks the arm 30 times a second while the model drives and decides when the run is stuck. |
| Expert rescue skill | The scripted pick skill from the previous post. When the watcher hands over, it moves the arm and finishes the pick. Its moves are the corrections. |
The difference matters for cost and scale. A person-run DAgger round costs a person's time per run. Here a round runs on its own: in the 50-run tests in this article, a person stepped in only a few times. The trade-off is that the expert rescue skill is only as good as its script, which is why the appendix covers improving it.
How is DAgger data prepared?
Recording, idle trimming, video conversion and upload work the same way as in the previous post. DAgger adds one rule: keep only the expert rescue skill's frames.
- Tag every frame. Each frame carries an intervention tag saying who was driving, the same format LeRobot's own DAgger mode uses.
- Keep only the expert rescue skill's frames. From each rescued run, only the frames where the skill was driving go into training. The model's own frames from before the takeover are dropped, because they would teach the stuck behavior again.
- Drop failed rescues. When the expert rescue skill also failed, the whole run is left out. Those runs are used to fix the skill instead.
- Trim idle frames. As before, every still stretch is cut to 3 frames, now in the human demos too.
| Dataset | What changed | Episodes | Frames |
|---|---|---|---|
| v3 | 47 demos, idle frames trimmed | 47 | 23,936 |
| v4 | Idle trim in the human demos too, plus 25 DAgger correction segments | 72 | 33,180 |
DAgger round 1: collecting data with the v3 model
Model: v3 from the previous post. Success rate: 0 of 30 (0%) on its own. 25 rescued by the expert rescue skill.
How a run works
- Stuck. The watcher calls a run stuck when the gripper tip moves less than 1.2 cm in 10 seconds, or when 45 seconds pass. A first version used 4 seconds and cut v3 off before it had a chance, so it was lengthened after 3 runs.
- Held or not. v3 closes the gripper to about the tube's width, so the joint reading alone can't tell a held tube from an empty grip. The watcher squeezes slightly: if the jaws stop on something, the tube is held.
- Takeover. The expert rescue skill starts from wherever v3 left the arm: rise, open, find the tube, line up, grip and lift. It gets two tries.
- Reset. The arm sets the tube down at a random spot and angle and starts the next run.
Video
One full run, sped up to 5 seconds (about 11×). v3 lines up and stops above the tube for about 41 seconds, the watcher calls it stuck, and the expert rescue skill takes over and lifts the tube.
A clean correction, sped up to 5 seconds (4×). v3 is stuck above the tube, and the expert rescue skill gets it on the first try.
A correction with a retry, sped up to 5 seconds (about 9×). The skill misses the first grip, rises, lines up again and gets it on the second try. Corrections like this are the most valuable DAgger data, because they show how to come back from a miss.
Dataset collected over 30 runs
| Outcome | Runs |
|---|---|
| v3 picked the tube on its own | 0 |
| Rescued by the expert rescue skill | 25 |
| Rescue failed too | 5 |
v3 went for the grab a few times, but never held on. Twice its jaws closed beside the tube and pushed it away, and those pushes caused most of the failed rescues, because they left the tube near the base or at a steep angle where the skill struggles.
Data: 30 episodes, 43,721 frames, 19,428 of them corrections from the expert rescue skill. Arm time: about 2 hours.
Model version 4: trained on the DAgger corrections
Dataset: 47 demos + 25 correction segments. Loss 0.016. Success rate: 8 of 50 (16%).
Dataset
The v4 dataset is the v3 data plus the expert rescue skill's correction frames from the 25 rescued runs. v3's own frames would just teach the stuck behavior again, so they are left out, along with the 5 failed rescues.
Training
| Dataset | 72 episodes, 33,180 frames |
| Machine | Runpod H100, $3.99/h, 3 h 12 min, $12.27 in total |
| Safety | Each checkpoint is uploaded to Hugging Face every 2,500 steps, so training can resume on any machine if the credit runs out |
Final loss was 0.016. The machine was deleted after the model was saved.
Video
v4 picks up the tube on its own, sped up to 5 seconds (about 1.7×). It reaches, lines up, closes on the tube and lifts it in 8 seconds, with no help. No version before it managed this once.
Results over 50 runs
| Outcome | Runs |
|---|---|
| v4 picked the tube on its own | 8 |
| Rescued by the expert rescue skill | 33 |
| Rescue failed too | 9 |
v4 succeeded on its own in 8 of 50 (16%) runs, up from 0 of 30 (0%) for v3. The watcher called a stall after 4 seconds without movement, the same rule used for every later model.
Lesson
Demos teach the ideal path. Corrections from the model's own mistakes teach the finish. The same model, retrained with 25 corrections, went from 0 lifts to 8 of 50 (16%). The expert rescue skill's quality sets the ceiling: wherever the skill itself fails, the model never gets a correction to learn from.
What comes next?
v4 trained only on corrections. Its own successful runs, the best examples of the task in its own style, are still unused. The next post grades every run and trains v5 on the model's own successes, weighted by how well each run went. That adds a first step of reinforcement learning (RL) on top of DAgger. In robot learning this simple, offline form is usually called reward-weighted or filtered behavior cloning: learn from your own runs, keeping and weighting the good ones.
Appendix: how was the expert rescue skill improved?
The expert rescue skill sets the ceiling: every correction the model learns from is one the skill pulled off. In the first rounds the skill itself missed too often, so it got a round of fixes. After every failed rescue, Claude went through all three cameras around the grab, named the cause, and fixed it before the next run.
Rescue success over time
Rescue success counts the runs where the model failed and the expert rescue skill then got the tube up.
| Tests | Skill version | Rescues needed | First try | Rescued |
|---|---|---|---|---|
| Tests 1–2, 70 runs | Original: home spot, search, stop-and-look line-up, fixed grip height. Touch-down, middle aim and the side camera added late in test 2 | 42 | n/a | 26 (62%) |
| Test 3, 12 runs saved | Continuous takeover | 7 | 5 (71%) | 5 (71%) |
| Test 4, 50 runs | Continuous takeover plus the fixes below | 44 | 39 (89%) | 42 (95%) |
| Test 5, 50 runs | All fixes, plus damping on the line-up | 45 | 42 (93%) | 43 (96%) |
Tests 1 and 2 are merged because both used the original skill and each was small on its own (27 and 15 rescues). Taken alone, test 2 dipped to 53%, partly from changes made mid-test that were fixed later.
How the final expert rescue skill works
The original skill went back to a home spot, searched for the tube and lined up in stop-and-look steps. That was slow, swung the arm side to side, and filled the training data with small hovering moves that the models then copied. The final expert rescue skill takes over the way a person would:
- Start where the model stopped, with no trip home and no wide search.
- In one move, rise to hover if lower, open the jaws and roll them across the tube.
- Line up while moving, in small steps, reading the wrist camera only after each step finishes. Reading mid-motion made it chase stale positions and overshoot.
- Never close unless lined up. If it isn't settled in time, finish with careful stop-and-look steps instead of grabbing blind.
- Feel down to the table, close and lift.
How problems were found: a third camera
The top and wrist cameras can't see height well, and most misses turned out to be height or reach errors. Chad added a webcam level with the table, facing the arm. It records all the time and is used only for analysis, never as input to the model, so the training setup is unchanged.
Three cameras: top camera on the boom, wrist camera on the gripper, and the side camera in the front right at table height. The top and wrist views are the model's inputs. The side view is for diagnosis only.
Two misses the other cameras couldn't explain. Left: the wrist view said lined up, but the fixed jaw came down on top of the tube. Right: the jaws held the tube by its thin end, but the grip reading was so low the skill called it a miss and let go.
Every fix, in order
| # | What went wrong | How it was found | Fix |
|---|---|---|---|
| 1 | Jaws closed above the tube near the base | Servo load | Touch-down: lower until the shoulder feels the table, rise 6 mm, close |
| 2 | Jaw landed on the tube | Side camera | New line-up target from grips that held |
| 3 | Aim jumped between the tube's ends | Wrist frames | Aim at the middle of the tube |
| 4 | Line-up swung after the wrist roll | Logs | Read the roll direction from the image |
| 5 | Thin-end grips counted as misses | Side camera | Lower held threshold |
| 6 | Detector locked onto the table | Wrist frames | Stricter color rules |
| 7 | Wrist camera froze | Frame counts | Side camera at 640x480 to share the USB |
| 8 | Slow, hovering rescues | Chad, hover time | Continuous takeover |
| 9 | Line-up overshot | Logs | Look only after each step finishes |
| 10 | Grabbed when not lined up | Logs, side camera | Never close blind |
| 11 | Aim slid to the cap | Side camera | Measure the middle from the cap end |
| 12 | Tube standing on its end | Side camera | Grip it halfway up |
| 13 | Jaws in line with the tube | Side camera | Re-check the crossing angle |
| 14 | Near the base: 3 cm off and too high | Chad, side camera | Hover above the real table |
| 15 | Jaw hit the base | Chad | Narrower opening near the base |
| 16 | Wrist camera hit the upper arm | Chad | Turn limit, and an edge case |
| 17 | Rescue crashed mid-run | Logs | Lock on the servo bus |
| 18 | Missed first tries in the training data | Rescue clips | Crop the missed first try |
Limits of the skill (edge cases)
Situations the expert rescue skill doesn't handle on its own. A person, or Claude driving the arm by hand, gets the tube back. These runs are kept out of the training data and out of the rescue success rate.
- Tube too close to the base. To cross the tube, the wrist has to turn, and this close to the base the turn swings the wrist camera into the arm's upper link, which can damage it. The skill now detects it, doesn't move, and the run loop waits for a person.
- Tube pinned under the arm. It ended up right under the parked wrist, where no grip could reach it. Claude used the closed gripper as a rake, rolling the tube outward until it pointed forward, then picked it up normally.
- Tube pushed out of reach. A failed rescue pushed it past the edge of the arm's normal reach. Claude stretched the arm out past the tube, lowered the closed gripper behind it and dragged it back into range (video below).
- Tube near the base, before the fixes. Every scripted grip there landed about 3 cm off and too high. Claude drove the arm one small move at a time with Chad coaching from a side view. The fixes above now handle most near-base tubes automatically, except the too-close case.
Pulling the tube back, filmed by Chad from the side, sped up to 5 seconds (about 2×). This is a recovery move between runs, not training data.