Removing Episodes From a LeRobot Dataset: Which to Cut

Define it by what the operator did, not by a score. Five kinds are worth naming separately.

The attempt that ran out the clock. The record loop runs while timestamp < control_time_s, where control_time_s is --dataset.episode_time_s, and breaks early only when the exit-early event is set. When it returns, the outer loop calls save_episode() unless the re-record event was set. An attempt that times out is written to disk exactly like a success, which is how a corpus fills with failures while the operator remembers a good session.

The mid-episode recovery. The gripper misses, the operator backs the arm off, re-approaches and completes the task. One episode, two attempts, and the second only makes sense given the first.

The idle tail. The arm sits still while the operator decides whether to stop the episode. The action column repeats and the policy learns that this state maps to no motion.

The teleop dropout. A leader-arm read stalls or a packet drops, and the action column carries a discontinuity that no physical motion produced.

The mislabelled task. The task string says "place the bolt in the left bin" and the demonstration puts it in the right one. Nothing in the numbers disagrees with itself. The label is wrong.

Set those against what detects each one and the list grows to seven: a grasp that closes on nothing has a signature of its own, and an attempt that plainly failed has none at all. Five of the seven are reachable only by proxy. Two need a human or a label.

Two failure modes are deliberately absent: camera movement mid-session, covered in camera placement for imitation learning along with the record-time control keys, and anything specific to synthetic or human-video sources, since where robot training data comes from changes which of these can occur at all.

Supervised imitation learns a mapping from observed state to demonstrated action and has no channel for "this action was a mistake". The 2010 reduction paper gives the bound: for naive supervised-learning imitation with expected per-step loss epsilon under the expert's own state distribution, the learned policy's cost is bounded by the expert's plus T squared times epsilon, quadratic in the task horizon T. Errors push you off the demonstrated distribution, and off-distribution states produce larger errors: A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. That is an argument about what supervised imitation is, not a measurement on a robot.

The measurement exists separately. The robomimic study built two corpora on the same tasks: a proficient-human set of 200 demonstrations from one experienced teleoperator, and a multi-human set of 300 demonstrations from six operators split into pairs described as better, okay and worse, 50 each. Transport is the exception: it needs two arms, so its proficient set is 300 demonstrations collected jointly by two proficient operators and its multi-human set is 300 from six pairings. Success rates below are averaged over three seeds, low-dimensional observations.

On the three single-arm tasks the multi-human corpus is the larger one, by 100 demonstrations, and it loses wherever the task is hard enough to lose on. The paper says so itself: results are lower for the multi-human datasets even though they contain 100 more demos. More episodes of uncontrolled quality is not a safe default. That is What Matters in Learning from Offline Human Demonstrations for Robot Manipulation.

LeRobot's own recording guidance says the same thing in one line. The documentation suggests at least 50 episodes with 10 per location, and then: "Keep the cameras fixed and maintain consistent grasping behavior throughout the recordings." That is a distribution constraint, and an acceptance rule is how you enforce it when a session produced something else. Our guide to how much data it takes to train a robot policy sizes programmes in accepted episodes and tells you to write the acceptance criteria into the contract. What follows is what goes in that clause.

Ten seconds of operator effort per episode beats every signal you can reconstruct afterwards. The operator watched the attempt; nobody reading the parquet files a week later has that, and the four criteria below exist only because the label was not captured.

LeRobot v0.6.1 gives you nowhere to put it: no success field in the episode schema, no success key in the record loop, no success prompt in the recording flow. The five default frame-level features are timestamp, frame_index, episode_index, index and task_index; everything else is a user feature. None of it is an outcome. Two real homes exist, and only two.

The first is the task string, which --operation.type modify_tasks rewrites per episode index, taking --operation.episode_tasks as a dict from episode index to task string, --operation.task_replacements as a dict from an existing task string to its replacement, or --operation.new_task as one string for everything, in that precedence. The documentation is emphatic that this works in place, rewriting meta/tasks.parquet, the task_index column, the episode metadata and total_tasks in meta/info.json, with --new_repo_id and --new_root ignored. The cost is overloading a field the policy conditions on.

The second is a file you own, one row per episode index, written by the operator at the end of each attempt. We use this one, because a label the training loop reads as language is a label it can learn from, which is not what an outcome flag is for. The key is fragile: delete_episodes reindexes survivors from 0, so a sidecar keyed on the pre-cut index describes a dataset that stops existing the moment you cut. Write it against a copy you never edit, or regenerate it from the cut log after every filter.

When the label was not captured, four signals are computable from what LeRobot already stores. Run them separately and look at what each caught before you union the lists. That is a reviewability argument rather than a measured one: how much the four overlap on a real session is something we have not measured and nothing we could open reports.

import statistics

from lerobot.datasets import LeRobotDatasetMetadata

meta = LeRobotDatasetMetadata("yourorg/pick_place")
lengths = meta.episodes["length"]
median = statistics.median(lengths)

# --dataset.episode_time_s defaults to 60, so this is the timer ceiling
timeout_frames = 60 * meta.fps

timed_out = meta.filter_episodes(lambda ep: ep["length"] >= timeout_frames)
suspiciously_short = meta.filter_episodes(lambda ep: ep["length"] < 0.5 * median)

print(f"{len(lengths)} episodes, median {median:.0f} frames at {meta.fps} fps")
print("ran out the clock:", timed_out)
print("far below median:", suspiciously_short)

import numpy as np

from lerobot.datasets import LeRobotDataset

# Build the dataset with no episodes= filter: the indices below are dataset-global
ds = LeRobotDataset("yourorg/pick_place")
actions = ds.select_columns("action")

def episode_actions(ep: int) -> np.ndarray:
    lo = ds.meta.episodes["dataset_from_index"][ep]
    hi = ds.meta.episodes["dataset_to_index"][ep]
    return np.asarray(actions.select(range(lo, hi))["action"], dtype=np.float32)

a = episode_actions(41)                      # (frames, 6) on an SO-101 follower
step = np.abs(np.diff(a, axis=0)).max(axis=1)
accel = np.abs(np.diff(a, n=2, axis=0)).max(axis=1)
gripper = a[:, 5]                            # gripper.pos, normalised 0 to 100

No source we opened gives a threshold for an idle run, a second-difference spike or a gripper toggle that would transfer to your arm, your control rate and your task. That is the state of the evidence rather than a gap in the method.

The procedure is the same for each: compute the per-episode value over the session, sort it, and look at the distribution. The distribution has a body and a tail, and the cut goes at the visible break rather than at a round number. Then hand-review a band on each side of the line before you commit: the five nearest above and the five nearest below.

lerobot-dataset-viz \
    --repo-id line4/weld_inspect \
    --root /data/lerobot \
    --mode local \
    --episode-index 88

The argument style differs from the edit tool: lerobot-dataset-viz takes --repo-id and --episode-index with hyphens, while lerobot-edit-dataset takes --repo_id with an underscore. And --root differs: for the viewer it is the parent directory the repo_id path hangs off, so the example above looks in /data/lerobot/line4/weld_inspect, while for lerobot-edit-dataset it is the dataset directory itself.

The band is what makes the rule defensible next month. A threshold with no review band is a number somebody picked; a threshold with ten reviewed episodes around it is a number somebody defended.

A recovery is contamination for one policy class and training signal for another, and the same published data shows both signs on the same task. In the robomimic operator-quality subsets, each single-quality group is 100 demonstrations (two operators at 50 each) and each two-group combination is 200. Compare the better-operator subset against that same subset plus the worse-operator one, on Square:

Plain behaviour cloning, mapping the current observation to the current action, is hurt by the multi-modality lower-quality demonstrations introduce: two different actions from similar-looking states, averaged. The recurrent variant carries history and turns the same data into coverage of states the proficient operator never visited.

So the rule is per task and per architecture. If the policy runs in a cell where a failed grasp means a human intervenes, cut the recoveries and keep the narrow distribution. If it runs unattended and must get itself out of trouble, recoveries are some of the most valuable frames you recorded. Decide before recording starts, because it changes what you tell the operator to do after a fumble: complete the task, or press the re-record key.

Compute the keep list with filter_episodes, then materialise it. The argument for materialising rather than passing indices at train time is not speed, which we have not measured either way: it is that the run then points at a named artefact you can put in the run record. Seven questions about what the delete operation does, answered from the v0.6.1 source.

Three rows deserve expansion. The _old directory is a directory move rather than a copy and the next in-place run deletes it first, so it is not a backup system. And the video row is a cost model: removing one episode from a file that also holds kept episodes re-encodes that file using the source dataset's own encoder settings, read back from meta/info.json, while removing everything in a file costs a copy. That follows from the layout described in how the LeRobot v3 layout stores episodes: episodes are packed into shared files with the metadata acting as the index, so a deletion is a rewrite rather than an unlink. And the guard row is literal: on a 200-episode dataset the valid indices are 0 through 199, so passing 200 raises Invalid episode indices: {200} rather than skipping it.

Now the perimeter case, where these details turn from trivia into constraints.

lerobot-edit-dataset \
    --repo_id line4/weld_inspect \
    --root /data/lerobot/line4/weld_inspect \
    --new_repo_id line4/weld_inspect_v2 \
    --new_root /data/lerobot/line4/weld_inspect_v2 \
    --operation.type delete_episodes \
    --operation.episode_indices "[12, 88, 141]"

Disk. The operation writes a fresh dataset rather than editing files in place, so it needs roughly two copies of the corpus on the volume for the duration. On a capture machine beside the cell holding this month's video, that is what fails at 3am.

There is no upstream. A public dataset can be re-pulled from the Hub after a bad edit. A corpus recorded on a regulated production line has no upstream copy, --push_to_hub is not a conversation you want with your security team, and your own snapshot is the entire safety net. Naming both --root and --new_root keeps the operation off the default cache path and leaves the source untouched. Take the snapshot before the first cut, not after the first mistake.

Statistics belong to the artefact. The delete path re-aggregates the surviving statistics into the new root, so the filtered dataset carries its own. Normalisation statistics are still their own failure class at training time, and what to check when a fine-tune returns 0% success covers the ones that bite there. Our own advice: if the corpus came out of a v2.1 to v3.0 conversion rather than being recorded natively, load it once and confirm the metadata resolves before editing.

Acceptance here means demonstrations entering training, a different object from acceptance of rollouts leaving evaluation, which has its own thresholds and statistics in how many trials it takes to evaluate a policy. Keep the two documents separate or the word does two jobs and neither well. Five things belong in this one.

The cut log has to use pre-cut indices: a log written against the filtered dataset describes a corpus that no longer maps to the one you reviewed. If the policy that ships was trained on 214 of 300 recorded episodes, the record of which 86 went and under which criterion is part of the evidence that the training set was controlled. In a regulated setting that is one CSV. Capture the before and after counts in the same log:

lerobot-edit-dataset --repo_id yourorg/pick_place --operation.type info
lerobot-edit-dataset --repo_id yourorg/pick_place_filtered --operation.type info

That operation is read-only and prints the repository ID, the episode count, the task count, the actual frame count, the average frames per episode, the average episode time in seconds, the fps and the size in MB. It is a summary, not a validator. --operation.show_features true dumps the feature schema alongside.

This week, do one thing. Run the info operation on the dataset you are about to train on, then compute the episode-length distribution with the lines above and put it on a screen. Within a minute you will know whether anything sat at the timer ceiling, and that is your first flagged list, produced without a threshold or a review meeting.