Isaac Lab vs MuJoCo vs Genesis for Robot Training Data

MuJoCo's README describes it as "Multi-Joint dynamics with Contact", a general purpose physics engine for robotics, biomechanics, graphics and machine learning. You install it with pip install mujoco, the bindings ship in the same package, and there is nothing above the physics.

Genesis self-describes as a simulation platform for physical AI development, and the platform claim is load-bearing. Its compiler layer, named Quadrants, was forked from Taichi in June 2025 and lowers Python kernel code to CUDA, AMD ROCm, Apple Metal, Vulkan, x86 and ARM64. Its solvers cover rigid bodies, FEM, MPM and particle methods with an explicit coupler, and it ships three renderers.

Isaac Lab is the one that is not a simulator. Its main-branch README says Isaac Lab requires Isaac Sim, which includes components under proprietary licensing terms. The README at the v3.0.0-beta2.patch1 tag names the workflows that need it, PhysX, RTX rendering, ROS and the URDF and MJCF importers, and adds that kit-less Newton workflows do not. Neither of those sentences is on main.

Read the bottom row first. MuJoCo's XML is the shared currency: Genesis loads MJCF through gs.morphs.MJCF, and Isaac Lab lists an MJCF importer among the workflows needing Isaac Sim. Your assets port; your data loop does not.

There is no speed row, deliberately. Genesis ships benchmark scripts under examples/speed_benchmark and publishes no FPS or throughput figure, and none of the three repositories publishes a measurement covering all three.

Isaac Lab's scripts/imitation_learning directory holds exactly three subdirectories: isaaclab_mimic, locomanipulation_sdg and robomimic. The isaaclab_mimic directory holds exactly three scripts: annotate_demos.py, consolidated_demo.py and generate_dataset.py, present on main and at the v3.0.0-beta2.patch1 tag alike.

# Isaac Lab: record human demos, annotate subtasks, then generate
./isaaclab.sh -p scripts/tools/record_demos.py \
  --task Isaac-Stack-Cube-Franka-IK-Rel-v0 \
  --device cpu --teleop_device keyboard \
  --dataset_file ./datasets/dataset.hdf5 --num_demos 10

./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/annotate_demos.py \
  --device cpu --task Isaac-Stack-Cube-Franka-IK-Rel-Mimic-v0 --auto \
  --input_file ./datasets/dataset.hdf5 \
  --output_file ./datasets/annotated_dataset.hdf5

./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
  --device cpu --headless --num_envs 10 --generation_num_trials 1000 \
  --input_file ./datasets/annotated_dataset.hdf5 \
  --output_file ./datasets/generated_dataset.hdf5

Those flags are real argparse definitions. generate_dataset.py declares --task, --generation_num_trials, --num_envs, a required --input_file, an --output_file defaulting to ./datasets/output_dataset.hdf5, plus --pause_subtask, --use_skillgen, --disable_dataset_compression and --external_callback. annotate_demos.py declares --task, --input_file, --output_file, --auto, --annotate_subtask_start_signals and --external_callback. Both then hand the parser to Isaac Lab's app launcher, which is where --device and --headless come from. The default extension names the container: this path writes HDF5.

MuJoCo gives you the other end: a model, a data struct, a step call and a reset call, with every layer above that yours.

import mujoco

model = mujoco.MjModel.from_xml_path("panda.xml")
data = mujoco.MjData(model)

while data.time < 1.0:
    mujoco.mj_step(model, data)

mujoco.mj_resetData(model, data)

Genesis sits with MuJoCo in scope: a scene, entities loaded from MJCF or primitives, a batched build, a step call. No record, annotate or generate stage. A code search for mimicgen returns no hits, and its four manipulation examples train and evaluate a grasping policy rather than generate demonstrations.

import genesis as gs

gs.init(backend=gs.gpu)

scene = gs.Scene(show_viewer=False)
scene.add_entity(gs.morphs.Plane())
scene.add_entity(gs.morphs.MJCF(file="xml/franka_emika_panda/panda.xml"))

# n_envs is the parallel batch; the value here is illustrative
scene.build(n_envs=64, env_spacing=(0.5, 0.5))

for _ in range(1000):
    scene.step()

The honest unit of comparison is the code you own. On the Isaac Lab path you configure a task, a Mimic-suffixed variant of it, and a set of flags. On the other two you write and maintain the teleoperation capture, the subtask boundaries, the replay harness and the episode writer. How much to generate is a separate question, answered in how much data a robot policy actually needs.

MuJoCo 3.4.0 shipped on 5 December 2025 with exactly one entry under bug fixes: a fix to the box-box distance computation. A correctness improvement by any reading, and also the boundary at which one LIBERO task stops meaning what its stored init states assume.

The task is libero_spatial task 5. Its stored init states place a bowl roughly 11 cm above a ramekin and rely on the settle inside the environment reset to drop it into position. Under mujoco 3.4.0 and later it does not seat: it ends up tilted on the rim, roughly half of it overhanging. The bowl's collision body is 40 box geoms and the ramekin's is 25, so the seating contact is pure box against box, exactly what the fix touched. That task's healthy behaviour depended on the pre-3.4.0 bug.

An open issue on the LeRobot repository connected this to policy scores. Its settle probe ran across engine versions on the same 50 init states.

# under mujoco 3.3.0 (premise holds)
{"mujoco": "3.3.0", "suite": "libero_spatial", "task_id": 5, "settle_steps": 250,
 "n_inits": 50, "n_slid_off": 50, "xy_cm_mean": 1.45, "tilt_deg_mean": 16.0}

# under mujoco 3.8.1 (premise broken)
{"mujoco": "3.8.1", "suite": "libero_spatial", "task_id": 5, "settle_steps": 250,
 "n_inits": 50, "n_slid_off": 50, "xy_cm_mean": 3.72, "tilt_deg_mean": 10.5}

Only one of the three signals discriminates. xy_cm_mean drift during the settle goes from 1.45 cm when the bowl seats to 3.72 cm when it perches on the rim. tilt_deg_mean runs the other way, 16.0 healthy against 10.5 broken, because a seated bowl in that geometry rests at an angle. n_slid_off reads 50 on both, the probe's 1 cm threshold being exceeded either way. An alarm wired to tilt passes the broken run; one wired to n_slid_off fails both.

What that did to measured success on the task, same init states, same checkpoints:

Four policies, four magnitudes. pi0.5 loses 4 points and the reinforcement-tuned OpenVLA-OFT loses 86, a spread about recovery behaviour rather than capability, worth holding when you compare the policies in that table. Whole-suite scores shift by up to roughly 9 percentage points from this one task.

Two details make it a physics problem, not a policy problem. Within each engine version the probe values are bit-identical across all 50 init states, so this is deterministic, and the boundary is exactly 3.4.0. And in a same-physics A/B, holding mujoco at 3.8.1 and replacing only the task-5 init file with a re-settled version, SmolVLA went from 28% back to 84% and OpenVLA-OFT with GRPO from 6 of 50 to 49 of 50, the other nine tasks unchanged.

The same issue lists three prior sightings nobody had connected: a bowl drifting about 1.6 cm after the init state is set under mujoco 3.6.0, one checkpoint scoring 98.1% under 3.1.0 against about 91% under 3.9.0, and a visual channel where floor rendering darkens from 3.3.3 onward. No shared cause until someone bisected the engine.

If a benchmark number goes into a document anyone acts on, sample size is the other half of the problem, covered in how many trials it takes to evaluate a robot policy.

The LIBERO fork does have a pin. A pull request merged on 10 June 2026 changed the dependency from mujoco>=3.0.0 to mujoco>=3.0.0,<3.9.0 and bumped the package from 0.1.3 to 0.1.4. That ceiling defends against an API break: 3.9.0 redesigned the contact margin and gap semantics, and 3.10.0 removed mjthread.h and changed mj_fullM from (m, dst, M) to (m, d, dst), part of an in-progress deprecation of mjData.qM in favour of mjData.M.

An upper bound guards the API and says nothing about behaviour. mujoco<3.9.0 admits every release from 3.4.0 through 3.8.x, and resolution lands on 3.8.1, the newest in the window and the one the probe above shows breaking the task. The pin protecting the API is the pin guaranteeing the affected physics.

Ten releases in roughly eight and a half months. At that cadence a lockfile regenerated on a new machine is a different experiment. Pin the engine to one version, and the layers on top of it.

# constraints.txt - the engine is pinned exactly, not upper-bounded
mujoco==3.3.7
robosuite==1.4.1
hf-libero==0.1.4

A constraints file describes intent, not the running process. Assert the value at the top of the evaluation job: the MuJoCo Python bindings expose the installed engine version as a module attribute set from the library's own version string.

import mujoco

EXPECTED_MUJOCO = "3.3.7"

if mujoco.__version__ != EXPECTED_MUJOCO:
    raise RuntimeError(
        f"Eval pinned to mujoco {EXPECTED_MUJOCO}, found {mujoco.__version__}. "
        "Box-box distance changed in 3.4.0; the task-5 init states no longer settle."
    )

Add a physics-level check above that: a version match proves the engine is the one you expected, not that the scene still behaves. Reset at fixed stored init states, settle for a fixed number of steps, record the horizontal displacement of the object the task depends on, and store those as golden values beside the dataset. The probe in that issue ran on CPU in seconds per version. Diff it in the job that produces the benchmark number.

This is the engine-level version of an argument we have made at the dataset level in a fine-tune that trains cleanly and rolls out at zero: the artifacts deciding your result are the ones nobody records.

The same repository carries a second open issue, smaller and likelier to cost you a day. On macOS and Windows, installing LeRobot's libero extra succeeds. Pip exits 0, no warning, no simulator. The extra, as it stands on main:

libero = [
  "lerobot[dataset]",
  "lerobot[transformers-dep]",
  "hf-libero>=0.1.4,<0.2.0; sys_platform == 'linux'",
  "lerobot[scipy-dep]",
]

The marker on the hf-libero line is the whole story. The resolver evaluates sys_platform == 'linux', finds it false, drops the dependency and installs the rest as asked. The failure appears later as a ModuleNotFoundError when the LIBERO environment is constructed, reading like a bug in your own factory.

The issue proposes raising at config time in one of two locations that already exist, the LiberoEnv config class or the make_env factory. No guard has been merged, so write one.

import importlib.util

if importlib.util.find_spec("libero") is None:
    raise RuntimeError(
        "The libero extra resolved without the simulator. "
        "hf-libero carries a sys_platform == 'linux' marker, so pip exits 0 "
        "on macOS and Windows with nothing installed."
    )

The same config carries the rest of the run manifest. LeRobot's LIBERO defaults are 20 fps, with an inline comment that it must match robosuite's default control frequency, stored init states with a hard reset, 360 by 360 rendering, two camera streams and a 7-dimensional action. Each is a default a future release can move.

Inside a regulated perimeter the deciding facts stop being features. MuJoCo and Genesis are single-licence Apache 2.0 dependencies from PyPI. The Isaac Lab path is four licence layers: BSD-3 for the framework, Apache 2.0 for isaaclab_mimic, proprietary cuRobo terms that isaaclab_mimic requires, and the Omniverse Licensing Terms Isaac Sim prompts you to accept the first time you build it. The demonstration-generation extension, the one capability making Isaac Lab interesting here, carries the proprietary runtime dependency.

Isaac Sim's LICENSE file says the repository is covered under Apache 2.0, and that building or using it requires additional components licensed under other terms, naming the Omniverse Kit SDK and 3D assets. The README carries an Apache-2.0 badge; GitHub's own licence detector classifies the repository as "Other". We did not open the Omniverse terms and will not tell you what they permit. The structural point stands without them: an interactive acceptance prompt inside a build step is exactly what a reproducible or air-gapped pipeline exists to eliminate.

The build environment is narrow too: Windows 11 or Ubuntu 22.04 and 24.04, GCC and G++ 11 on 24.04, and a 260-character Windows path-length limit that causes build errors. RTX rendering sits among the workflows needing Isaac Sim, so RTX-class NVIDIA hardware comes with that path.

Then cadence, where the compatibility table repays a second reading.

Eight rows, and the third column carries the point. A default clone puts you on main, whose README maps main and v2.3.X to Isaac Sim 4.5, 5.0 and 5.1 and never mentions Isaac Sim 6. Those rows appear only at the 3.0 beta tag, whose README calls that branch a development branch for Isaac Sim 6.0, under active development and liable to breaking changes. The newest release is v3.0.0-beta2.patch1, from 2 July 2026, and the lines diverge structurally: source/ holds 6 extensions on main and 15 at the beta tag, where the backend splits into named physx, ov, ovphysx and newton extensions.

None of that makes Isaac Lab a bad choice. It makes it a choice you plan a migration for, on a vendor's schedule rather than yours.

The licence stack is one perimeter question. The second is records. If a qualification file says a policy scored 80% on a named benchmark, and whether that reads 80% or 28% depends on which engine version the resolver picked on install day, the engine version is part of the record. It belongs in the manifest beside the dataset revision and the checkpoint hash, emitted by the harness. The physical AI pillar applies the same discipline to the rest of a robot deployment's evidence.

The install-time silence is the third. Behind a perimeter nobody can install again from the open internet and compare, so the config-time assertion is what catches a mirrored index that resolves cleanly and omits the simulator.

Choose by the layer you are missing and by the stack already holding your assets.

Six situations, and the fifth row is a hard constraint rather than a preference. LeRobot ships simulation environment wrappers for six suites and none for Isaac Lab or Genesis. Its LIBERO wrapper imports the offscreen render environment from the LIBERO package, the robosuite and MuJoCo path. Not every wrapped suite is MuJoCo-backed, and one documents a SAPIEN-based API instead, but if your stack is LeRobot then MuJoCo is the engine you are already coupled to.

The MuJoCo repository dates to August 2021, Genesis to October 2023, the Isaac Sim repository to May 2025. Genesis is the one of the three whose README lists FEM, MPM and particle solvers alongside rigid bodies. Write your own regression probe before you depend on its output.

This week, do one thing that takes about an hour. Take the last three benchmark numbers your team reported for a simulated policy and find the engine version that produced each: not the range in the dependency file, the version installed in that run. If the logs do not have it, add the version assertion above to the evaluation entry point. Then re-run one under the pinned version and see whether the number you already told someone is still true.