Behaviour cloning trains only on states the expert visited, so small errors push a deployed policy into states it never saw, and DAgger fixes that by collecting expert labels on the states the learner actually reaches. The 2025 RaC paper turned that into a two-rule protocol (recover to a familiar state, then correct the current sub-task, then end the episode) and reported beating the prior state of the art with 10x less data collection time and samples on three real bimanual tasks. On shirt hanging it reached 78.3% with 5 hours of data, against about 89 hours for 75.0% in a prior system. Switch from fresh demos to correction rounds when the policy completes some attempts but fails in the same few states and more demos of the same scene have stopped helping. LeRobot v0.7.0 records those rounds with lerobot-rollout --strategy.type=dagger and tags human frames intervention=True, but lerobot-train does not read the tag and the documented workflow removes it before merging. Its mid-run Hub pushes ignore --dataset.push_to_hub=false. Start with one small corrections-only round on a local root, with egress blocked at the rig.
Your first policy works some of the time. It picks the part, carries it, and then on one attempt in three it clips the fixture edge at the same angle, or stalls with the gripper half closed in the same spot, and the run is over. The obvious move is to book the operators for another week of clean teleoperated demos of the same scene. DAgger imitation learning is the argument for doing something else with that week: let the policy run, and record the operator taking over exactly where it goes wrong.
The argument has numbers behind it now. A 2025 method called RaC (Recovery and Correction) added a disciplined correction phase after ordinary imitation learning and, across three real bimanual tasks and one simulated task, reported beating the prior state of the art with 10x less data collection time and samples (arXiv 2509.07953). The catch is in the word disciplined. Intervening whenever something looks wrong produces a different, weaker dataset than the protocol the paper ran.
LeRobot v0.7.0, released 5 November 2026, ships the recording half of that loop as lerobot-rollout --strategy.type=dagger, and its source docstring says it "Implements the RaC paradigm". It records and tags corrections. Nothing in training reads the tag. When to switch from fresh demos to correction rounds is a decision you can make from signals on your own cell, and on that release the mechanics of a round also decide whether footage of your cell leaves the building.
What DAgger imitation learning fixes: compounding error vs behavior cloning
Behaviour cloning is supervised learning on demonstrations: observation in, expert action out. It trains only on states an expert visited. Deployed, the policy makes a small error, the error moves it to a state slightly outside the demonstrations, its next action is a little worse because it has never seen that state, and the deviation grows. This is distribution shift, and in imitation learning it compounds, because each action decides the next observation.
The paper that introduced DAgger, submitted in November 2010 and published at AISTATS 2011, put the cause in one line: in sequential prediction problems like imitation learning, future observations depend on previous actions, which violates the independence assumption plain supervised learning relies on (arXiv 1011.0686). Its fix is an iterative loop framed as no-regret online learning, and it outperformed earlier approaches on two imitation learning problems and a sequence labelling benchmark.
The DAgger algorithm, in plain terms: train on demonstrations, run the learner, have the expert label the correct action in the states the learner actually visited, add those labelled states to the dataset, retrain, repeat. Behaviour cloning's dataset only ever contains the expert's states. DAgger's dataset grows toward the learner's states, which are the ones where it fails. Both use the same loss. The only difference is which states are in the data.
That is also why more clean demos stop paying at some point. A fresh demo records an expert starting from a reset scene and doing the task well, so it adds more coverage of the states the expert visits. If your policy fails in states the expert never enters, the hundredth clean demo covers the same ground as the first fifty.
DAgger, HG-DAgger and RaC: how human-in-the-loop imitation learning changed
The three methods differ in one question: who controls the robot while the fix is being recorded.
DAgger: labels without control
Original DAgger asks the expert what they would do in each visited state while the learner keeps driving. For a software expert that is fine. For a human it means labelling actions for a robot they are not steering.
HG-DAgger: the human takes control
The HG-DAgger paper names the two problems with that (arXiv 1810.02890): labelling without being in control can decrease safety, and human labels degrade because of perceived actuator lag. Its answer is to let the human take control when needed and learn from those segments. It also learns a safety threshold on a model-uncertainty risk metric. It was evaluated on autonomous driving, one simulated and one real-world task, where it improved over DAgger and behaviour cloning. Manipulation results come from later work.
RaC: recover, correct, stop
RaC is a training phase that runs after imitation learning pre-training. The paper's protocol starts with a round of full demonstrations (which can be an existing community dataset), trains and evaluates a first policy, then scales data with rounds of intervention and retraining. When failure looks imminent, the operator intervenes under two rules. The rules exist because recoveries are rare when nobody asks for them. On a 1% sub-sample of the DROID dataset, the researchers found that only 3.68% of episodes contain at least one recovery and 16.58% at least one correction. Their own HG-DAgger-style intervention data skewed toward corrections with scarce recovery segments. Without an explicit rule, the backing-out step is the one that goes missing. The headline comparison is the paper's Table 1 for shirt hanging. A prior 217M-parameter system used about 89 hours (5,345 expert demos) for 75.0% full-task success; a prior 4B-parameter vision-language-action model used 116 hours of demos plus vision-language data for about 63.6%, a figure the RaC authors estimated from a diagram in that paper; RaC's 368M-parameter model used 5 hours of expert, recovery and correction data for 78.3%. Read that as the authors' cross-paper comparison under similar but not identical setups, not a controlled experiment. The imitation objective is unchanged in RaC: the gains come from what is in the data. One finding is easy to misread. The paper reports that the trained policy's performance scales linearly with the number of recovery maneuvers it performs at test time. It is a statement about the trained policy's behaviour, not a rule that more collected corrections buy proportionally more success.
- 1. Pair recovery with correction. First bring the robot back to a familiar, in-distribution state, then push the current sub-task forward to completion.
- 2. Terminate after intervention. Once the intervention ends, end the episode. Letting the policy continue would put later sub-tasks under a state distribution induced by policy and human together, which may not help the policy under its own distribution and costs samples.
| Method | Who controls the robot during the fix | What gets recorded | Evaluated on |
|---|---|---|---|
| DAgger (2010) | The learner | Expert action labels for the learner's states | Two imitation learning problems and a sequence labelling benchmark |
| HG-DAgger (2018) | The human, after taking over | The human's takeover segments | Simulated and real-world autonomous driving |
| RaC (2025) | The human, under two rules | A recovery segment, then a correction segment, then the episode ends | Three real bimanual tasks and one simulated assembly task |
When to use DAgger instead of more robot demonstrations
No primary source gives a threshold: no operator-hour count, intervention rate or episode number at which to switch. RaC's 5 hours is its total for one task, not a switching rule. The signals below are observable on your cell, and they are what the decision should rest on.
Switch to corrections when three things are true together. The policy completes some attempts, so it already does the task's main motion. It fails in states you can name and reproduce, mid-task and recoverable, rather than at the first grasp. And adding demos of the same scene has stopped moving the success rate, which you read off the training-subset curve described in how much data a robot policy needs. When operator time on the rig is the cost that binds, those three signals point the next week at correction rounds.
Stay with fresh demos when the policy fails everywhere or from the first step: there is no competent policy to correct yet. Also when the failure is a new object, lighting or fixture the data never contained, because that is a coverage gap and corrections only cover it slowly. Corrections are still teleoperation data, collected on-policy, so the wider choice between teleop, simulation and human video in our comparison of robot data sources still applies.
The last two rows are about the rig, not the data. A correction round lets the policy drive the real arm next to real fixtures, with an operator ready to stop it. If the site's risk assessment does not cover that, the answer is demos regardless of the curve. And a jerky handover contaminates the data you are paying for: a September 2026 study of DAgger-style post-training for high-DoF dexterous manipulation found that mismatch at human takeover creates command discontinuities that contaminate corrective trajectories (arXiv 2609.19666).
| Situation on the cell | What to record next | Recording mode |
|---|---|---|
| Policy fails from the first step or on most attempts | Fresh demos | lerobot-record, not rollout |
| New object, lighting or fixture absent from the data | Fresh demos of the new condition | lerobot-record, not rollout |
| Completes some attempts, fails in the same few states, subset curve flat | Correction rounds | Corrections-only (default) |
| Failures happen mid-task and the arm can be backed out of them | Correction rounds | Corrections-only (default) |
| You need the intervention rate per round or the autonomous lead-up to each failure | Correction rounds | Continuous (record_autonomous=true) |
| Repeatable failures on the old setup, plus a new variant being introduced | Both | Corrections-only for the old setup, lerobot-record for the new |
| Leader-to-follower handover not yet reliable on this rig | Fresh demos until the handover is fixed | Not yet |
| Safety case does not allow the policy to move unsupervised near the fixture | Fresh demos | Not applicable |
LeRobot DAgger: running a correction round with lerobot-rollout
At v0.7.0 the strategy is a four-transition state machine. Space moves AUTONOMOUS to PAUSED and back. Tab moves PAUSED to CORRECTING and back. There is no AUTONOMOUS to CORRECTING transition, so taking over is always Space then Tab; a lone Tab while the policy runs is dropped. Enter is the upload key, and ESC stops the session. While paused, nothing is recorded and the robot holds position.
lerobot-rollout \
--strategy.type=dagger \
--strategy.num_episodes=40 \
--policy.path=outputs/pretrain/checkpoints/last/pretrained_model \
--robot.type=so101_follower \
--robot.port=/dev/ttyACM0 \
--robot.cameras='{front: {type: opencv, index_or_path: "/dev/video0", width: 640, height: 480, fps: 30}}' \
--teleop.type=so101_leader \
--teleop.port=/dev/ttyACM1 \
--dataset.repo_id=cell_a/rollout_corrections_r1 \
--dataset.root=/data/lerobot/rollout_corrections_r1 \
--dataset.single_task="Place the bracket in the fixture" \
--dataset.fps=30 \
--dataset.push_to_hub=false \
--dataset.streaming_encoding=trueControls and requirements
The keyboard defaults live under --strategy.keyboard.*. Set --strategy.input_device=pedal to use a three-button foot pedal instead (evdev codes KEY_A, KEY_B and KEY_C for pause, correct and upload, on the device at --strategy.pedal.device_path), so the operator keeps both hands on the leader arm. The pedal has no stop action; ESC is keyboard only. The configuration enforces five things. A teleoperator is required (--teleop.type). A dataset is required (--dataset.repo_id), and the name after the slash must start with rollout_ or the command raises; it then appends a _YYYYMMDD_HHMMSS stamp unless --dataset.no_stamp=true. A simulated robot is rejected, so correction rounds cannot be rehearsed in simulation through lerobot-rollout at this release. --interactive=true is not supported. And --dataset.fps must equal --fps, which defaults to 30, or the config raises.
Handover
With smooth_handover on (the default), pausing an actuated leader such as the SO-101, one that accepts position feedback, drives the leader to the follower's last commanded pose over 2.0 seconds with its torque on. The function that does it sets no torque or velocity cap of its own, so the operator protocol is simple: hands off the leader until it stops moving. Tab then releases the leader's torque for the correction, and ending the correction re-enables it. Teleops that cannot be driven get the opposite treatment: the follower slides to the teleop's pose over 1.0 second before the correction starts. The project's guide lists only actuated leaders as supported, but the code handles non-actuated teleops by moving the follower instead. Two caveats from the source. Disable smooth_handover for clutch-style teleops such as VR controllers, which re-reference at the current pose on engage; that is RaC's own interface type. And a TODO in v0.7.0 notes that if your processor pipeline renames action keys, the leader move silently does nothing. With a matched SO-101 leader and follower the keys are identical. With a rename map, test the handover before the operators start, because that is the table's "not yet reliable" row.
The command
This runs a corrections-only round of 40 interventions against a pretrained checkpoint, writing to a local folder. The camera key and paths are placeholders and must match what the policy was trained on. --dataset.push_to_hub=false here stops only the final push at teardown. Pressing Enter still calls a Hub push in this mode, which is why the regulated-perimeter section below blocks egress at the host as well.
Mapping RaC's rules onto the keys
LeRobot does not enforce RaC's rules; the operator does. For Rule 1, the operator's first movement after Tab backs the arm out to a configuration the policy has seen, and only then completes the sub-task. The guide's own advice matches: intervene before an irreversible mistake, on hesitation or on deviation, recover to a familiar configuration, and make corrections confident, clean and complete through the current sub-task. For Rule 2, press Tab to end the correction (which saves it as one episode), then reset the scene for a fresh attempt instead of pressing Space to let the policy resume from the state the human left it in. If the policy resumes, any later correction in that attempt becomes a new episode recorded under the mixed distribution Rule 2 exists to avoid.
Corrections-only vs continuous recording in LeRobot's dagger mode
The record_autonomous flag picks between two different datasets.
In corrections-only mode --strategy.num_episodes counts corrections and falls back to --dataset.num_episodes, whose default is 50. Watching time is spent and leaves no frames, so the dataset tells you nothing about how long the policy ran before each failure.
Continuous mode ignores num_episodes entirely, even though the project's own continuous-mode example passes one. Episodes rotate on a duration estimated from the smallest camera's resolution, the fps and a target video file size (200 MB by default) at a conservative 0.1 bits per pixel. For one 640x480 camera at 30 fps that formula gives about 1,820 seconds, roughly 30 minutes per episode, however many task attempts happen inside it.
Neither mode gives you one episode per task attempt, autonomous lead-up plus corrections. That gap is GitHub issue 4626, open as of this writing, and the pull request proposing an episodic DAgger mode is unmerged. Until it lands, log per-attempt success and interventions per attempt outside LeRobot: a spreadsheet row per attempt is enough.
Use corrections-only for RaC-style rounds; it is the mode whose episodes match Rule 2. Use continuous when you need the autonomous context, for example to show a reviewer the intervention share falling from round to round.
Corrections-only (record_autonomous=false, default) | Continuous (record_autonomous=true) | |
|---|---|---|
| What is recorded | Correction windows only | Autonomous and correction frames |
intervention values present | True only | True and False |
| Episode boundary | One episode per correction, saved when Tab ends it | Rotated by an estimated duration, deferred while a correction is active |
| Stop condition | num_episodes corrections, ESC, --duration or shutdown | ESC, --duration or shutdown; num_episodes is ignored |
| Mid-run upload trigger | Upload key (Enter or KEY_C) | Every upload_every_n_episodes saved episodes (default 5) |
| Encoding | Streaming optional | Streaming forced on, with a warning |
How lerobot-train handles DAgger intervention data
The dagger strategy adds a boolean intervention feature of shape (1,) to the dataset. At v0.7.0, lerobot-train never reads it: a search for intervention across the training scripts and the dataset code returns nothing, and the built-in sample weighting supports only rabc (progress-file based) and uniform. Upweighting human frames is not a supported option, and the project publishes no result showing it helps, so do not build a recipe on it.
The project's guide runs the loop as: train a base policy on demos, collect with the dagger strategy, remove the intervention column, merge with the demos (merge requires identical features), fine-tune the step-one policy on the merged set, redeploy, repeat. Do the removal on a derived copy and keep the tagged session folder untouched, because it is your only record of which frames were human.
lerobot-edit-dataset \
--repo_id cell_a/rollout_corrections_r1_20261109_141500 \
--root /data/lerobot/rollout_corrections_r1 \
--new_repo_id cell_a/corrections_r1_no_intervention \
--new_root /data/lerobot/corrections_r1_no_intervention \
--operation.type remove_feature \
--operation.feature_names "['intervention']"The repo id carries the stamp lerobot-rollout appended at creation; yours will differ. lerobot-edit-dataset defaults to push_to_hub=False. The merge itself, and what it rewrites, is in what a LeRobot dataset merge actually breaks. Keeping recoveries that operators made inside ordinary teleop demos is a separate question, covered in removing bad episodes from a LeRobot dataset.
At this version the tag is for your own analysis: where corrections cluster in the task, how long they run, whether the share of human frames falls between rounds. Whether a round actually helped is an evaluation question. Pick the round's checkpoint with the checkpoint selection guide and size the before-and-after comparison with how many trials a policy evaluation needs.
Running LeRobot DAgger offline inside a regulated perimeter
Correction rounds happen on the production or pilot cell itself, with a deployed checkpoint on the rig. Every saved episode is camera footage of your facility, your fixtures and your operators, plus joint trajectories. At v0.7.0 DatasetRecordConfig defaults to push_to_hub=True with private left at None, and the dagger strategy's mid-run pushes never read the flag: every 5 saved episodes in continuous mode, on Enter or KEY_C in corrections-only mode. Only the teardown push honours --dataset.push_to_hub=false. A failed background push is caught and logged as Background push failed, so on a sealed host the session continues. The same behaviour across the other rollout strategies is inventoried in which LeRobot commands call Hugging Face.
That makes the rig a network decision, not a flag decision. Set --dataset.root explicitly, pass push_to_hub=false, and block outbound traffic at the host, so an operator's accidental Enter is a log line and not an upload.
The decision itself also favours keeping the work in-house. A correction round has to run on the real cell, with the real fixtures, and the strategy refuses simulated robots. It cannot be shipped to a remote teleoperation service without shipping the cell's footage with it. The asset the round captures is the judgement of the operators you already have: when to step in, how far to back out, what clean completion looks like on this line. Budget their attention accordingly. In corrections-only mode an operator watches the policy for long stretches that leave no frames, and that watching is part of the cost. The rest of the physical AI pillar covers the data pipeline around it.
How to run your first DAgger correction round this week
Take the policy you have. Run one small corrections-only round on a local --dataset.root with --dataset.push_to_hub=false and egress blocked at the host, and brief the operators on the two rules: back out first, finish the sub-task, end with Tab, reset the scene. Keep a row per attempt with its outcome and number of interventions. Then look at where the corrections cluster. If they bunch in two or three states, retrain on the merged set and run another round. If they are spread everywhere, the policy needed more demos, and you found that out for the price of one session.
FAQ
Quick answers to the questions this post tends to raise.
