Joint Space vs Task Space: Which Robot Action Space to Use

DROID 1.0.1 refuses to choose. The LeRobot conversion's meta/info.json declares a Franka, 95,600 episodes and 27,612,581 frames at 15 fps, with eight action columns side by side in every episode.

The state side repeats the split, down to a flat observation.state at 8. Two stacks consume those episodes:

# OpenVLA: prismatic/vla/datasets/rlds/oxe/utils/droid_utils.py
dt = trajectory["action_dict"]["cartesian_velocity"][:, :3]
dR = trajectory["action_dict"]["cartesian_velocity"][:, 3:6]
trajectory["action"] = tf.concat(
    (dt, dR, 1 - trajectory["action_dict"]["gripper_position"]),
    axis=-1,
)

# openpi: src/openpi/policies/droid_policy.py
state = np.concatenate([data["observation/joint_position"], gripper_pos])
# ...
# Only return the first 8 dims.
return {"actions": np.asarray(data["actions"][..., :8])}

OpenVLA builds a 7-dimensional delta end-effector action out of the Cartesian velocity column and an inverted gripper. openpi never touches a Cartesian column: it builds an 8-dimensional joint-plus-gripper state and returns 8 dimensions of action. So "what action space is DROID in" has no answer. The action space is a consumer decision, made in a transform file by whoever trained the checkpoint.

OpenVLA states the taxonomy in four lines of prismatic/vla/datasets/rlds/oxe/configs.py.

class ActionEncoding(IntEnum):
    EEF_POS = 1             # EEF Delta XYZ (3) + Roll-Pitch-Yaw (3) + Gripper Open/Close (1)
    JOINT_POS = 2           # Joint Delta Position (7) + Gripper Open/Close (1)
    JOINT_POS_BIMANUAL = 3  # Joint Delta Position (2 x [ Joint Delta Position (6) + Gripper Open/Close (1) ])
    EEF_R6 = 4              # EEF Delta XYZ (3) + R6 (6) + Gripper Open/Close (1)

Grepping it for the enum returns 67 EEF_POS against 3 JOINT_POS across 70 dataset configs. That counts registered configurations, not the training mixture, and the shipped head is 7-DoF delta end-effector plus gripper either way. Fine-tuning it on joint-space data is not a flag: it takes new registry entries, a new transform, a mixture entry and a training config.

openpi went the other way, per adapter. Its DROID adapter is joint space at 8 dimensions. Its ALOHA adapter documents state: [14] and actions: [action_horizon, 14], also joint. Its LIBERO adapter returns data["actions"][..., :7], and LIBERO's environment wrapper loads controller="OSC_POSE", which makes that column delta end-effector.

GR00T declares the layout per embodiment in a modality.json, and its three shipped examples declare three different ones.

"action": {
  "eef_9d":           { "start": 0,  "end": 9  },
  "gripper_position": { "start": 9,  "end": 10 },
  "joint_position":   { "start": 10, "end": 17 }
}

That is the DROID sample: 17 dimensions carrying both spaces at once. The LIBERO example declares six named pose dimensions plus a gripper; the SO100 example declares single_arm over 0, 5) and gripper over [5, 6). Which checkpoint to fine-tune is settled in [our comparison of OpenVLA, pi0, SmolVLA and GR00T N1; choosing it chooses the action space too.

That February 2026 study is the only controlled experiment here. It trained more than 500 models on more than 2,000 demonstrations across single-arm and dual-arm AgileX platforms, a single-arm AIRBOT and 10 of RoboTwin 2.0's 50 simulated tasks, with initial conditions from a 6x6 grid and three trials of ten rollouts each. Scores are progress scores, standard errors 1.4 to 4.8 on the overall averages and as wide as 6.2 on the simulation rows.

Read the columns, not the cells. Moving left to right within a space always helps, and in five of six rows it helps more than switching space. The absolute level collapses in simulation while the ordering survives. A 79.7 against an 88.0 only means something if your evaluation can resolve it, which is a question of how many trials it takes.

The spatial finding is conditional. Joint space wins in domain and its advantage grows with data and compute, so a team collecting on one arm keeps widening the gap; how much data that takes is its own question. Under cross-embodiment training and transfer from a pi0 foundation model, task space becomes more advantageous and sometimes surpasses joint space, because it abstracts away robot-specific kinematics. Delta still wins in both.

Delta is not one thing, and treating it as one is how an anchor mismatch reaches a training run. Three anchors ship in code covered here: an offset from the previous predicted action (step-wise), from the state at the start of the chunk (chunk-wise), and from a pose latched at episode reset.

The difference is not stylistic. Step-wise delta must be integrated to recover a pose, so the decoding matrix over a chunk of length k is the k by k lower-triangular matrix of ones and the worst-case error bound grows with k, roughly (2k+1)/pi times the prediction-noise bound. Chunk-wise delta and absolute actions decode through the identity and stay flat. Measured, that was a gap upwards of 10% on average in favour of chunk-wise.

Horizon couples to the anchor. Every policy was trained with a chunk of 60, described as 2 seconds at 30 Hz, and the execution horizon was grid-searched from 15 to 60: absolute benefited from a significantly longer horizon while delta peaked at a shorter one, so the settings used thereafter were 30 for delta and 60 for absolute. Copying a horizon from work that used the other anchor gives back the gain you switched for.

LeRobot ships the third anchor in its RoboTwin environment's ee mode, where the 16-dimensional prediction is composed onto the end-effector pose latched at reset: translation added, rotation composed as the initial rotation times the predicted one, both quaternions renormalised. Neither step-wise nor chunk-wise.

The ambiguity is live rather than settled. LeRobot issue #4091, filed 2026-07-20 and still open with no reply, reports that PI05Config.use_relative_actions defaults to False for the pi05_libero checkpoints while the LIBERO environment is configured for delta control, and asks whether those checkpoints natively emit deltas. When an anchor mismatch bites it presents as a fine-tune that converges cleanly and scores 0% on the robot, where the relative-action tooling is covered.

The framing hides an asymmetry. Joint space has one convention to agree on: units and motor order. Task space has three, each failing silently.

The frame case is concrete. velocity_act_to_wrist_frame rewrites the same base-frame 6-dimensional velocity action as a 9-dimensional wrist-frame one, rotating translation by the inverse frame rotation and conjugating the rotation delta. Identical motion, different numbers, both shipped in one file. The wrist frame moves with the arm, so the choice interacts with where you put the cameras: a static view against a moving action frame leaves the transform for the policy to learn.

Nothing converts between them for you. An Isaac-GR00T issue open since 2026-05-18 reports a dataset storing tool poses as 7-dimensional quaternions against a pipeline expecting the 9-dimensional rot6d layout, with no official converter.

The gripper convention rides along quietly. OpenVLA's base-frame DROID transform writes 1 - gripper_position into the last action channel, a sign flip that commands open where the source said close, while its wrist-frame transform passes the same channel through untouched. Fine-tune across that boundary and the arm politely releases every object it approaches.

The content-farm version of this comparison says Cartesian control avoids singularities. It is exactly inverted. Task space is what introduces a potentially ill-conditioned pseudo-inverse Jacobian into the control transform; joint space is numerically stable by construction, and pays for it by making the policy regress vision onto a highly non-linear configuration space. That is the whole tradeoff, from the study's appendix.

LeRobot v0.6.1 does not solve IK analytically. src/lerobot/model/kinematics.py wraps placo and configures a soft weighted quadratic-programming task.

def inverse_kinematics(
    self,
    current_joint_pos: np.ndarray,
    desired_ee_pose: np.ndarray,
    position_weight: float = 1.0,
    orientation_weight: float = 0.01,
) -> np.ndarray:
    ...
    self.tip_frame.configure(self.target_frame_name, "soft", position_weight, orientation_weight)
    self.solver.solve(True)

Orientation is weighted a hundredfold below position by default, so where the two conflict the solver trades orientation away with no exception and no log line. The default is deliberate: the InverseKinematicsEEToJoints docstring says to set the weight to 0.0 for position-only IK on under-actuated arms, and that a small nonzero weight gives soft-orientation IK on the 5-DOF SO-101, where the wrist tracks orientation only partially. That follower has five arm joints plus a gripper, so a full 6-DoF pose command on it is a request.

The ordering of the safety steps matters more than the solver does. EEBoundsAndSafety clips the commanded position into end_effector_bounds and rate-limits the per-frame step to max_ee_step_m, default 0.05. It runs before InverseKinematicsEEToJoints, so it bounds where the tool centre point may go and says nothing about the joint excursion needed to get there. Near a wrist singularity a small Cartesian step can demand a large wrist_roll move, and the cap that would catch that, max_relative_target, defaults to None.

There is no flag for this. RecordConfig in v0.6.1 has exactly ten fields, and none of them names an action space. Nor is there a robot type: the so100_follower_end_effector that existed at 0.3.3 is absent from v0.6.1, with the issue asking for it back unanswered since 2026-04-23. Current practice is a Python script composing processor steps.

from lerobot.model.kinematics import RobotKinematics
from lerobot.robots.so_follower.robot_kinematic_processor import (
    EEBoundsAndSafety,
    ForwardKinematicsJointsToEE,
    InverseKinematicsEEToJoints,
)

follower_kinematics_solver = RobotKinematics(
    urdf_path="./SO101/so101_new_calib.urdf",
    target_frame_name="gripper_frame_link",
    joint_names=list(follower.bus.motors.keys()),
)

steps = [
    EEBoundsAndSafety(
        end_effector_bounds={"min": [-1.0, -1.0, -1.0], "max": [1.0, 1.0, 1.0]},
        max_ee_step_m=0.10,
    ),
    InverseKinematicsEEToJoints(
        kinematics=follower_kinematics_solver,
        motor_names=list(follower.bus.motors.keys()),
        initial_guess_current_joints=True,
    ),
]

Those pipelines go into record_loop as teleop_action_processor, robot_action_processor and robot_observation_processor, and the on-disk schema becomes ee.x through ee.wz plus a gripper channel, produced by the pipeline, not the robot. That example sets no max_relative_target on the follower, so nothing caps the joint excursion the solver emits.

Even with identical leader and follower arms, the leader's joints go through forward kinematics to a pose and back through IK onto the follower, so on a five-joint arm you record an orientation the follower only partially realised. Teleoperation is also the only data source giving you joint-level action labels on your own embodiment, one more thing to weigh against simulation and human video.

The decision is not close once the regime is written down. One fixed embodiment, a growing demonstration set, evaluation on your own hardware: joint space, chunk-wise delta. A cross-embodiment prior to fine-tune, or a second arm to move onto later: task space, chunk-wise delta, and budget for the frame and rotation conventions above. An under-actuated arm such as the 5-DoF SO-101: joint space, unless you have accepted position-only IK.

Whichever you pick, verify what your action column contains. The test is the one a public LeRobot audit ran across 6,047,603 rows.

import numpy as np
import pandas as pd

df = pd.read_parquet(path)          # one episode from your dataset
act = np.stack(df["action"].to_numpy())
sta = np.stack(df["observation.state"].to_numpy())

n = min(len(act) - 1, len(sta) - 1)
exact = np.all(act[:n] == sta[1 : n + 1], axis=1).mean()
close = np.isclose(act[:n], sta[1 : n + 1], atol=1e-4).all(axis=1).mean()

print(exact, close, np.percentile(np.abs(act), 99, axis=0))

# exact near 1.0 -> an absolute next-state target; close near 1.0 -> absolute
# with a controller offset; both near 0 with a small p99 -> deltas

Every non-terminal action[t] in the RoboTwin conversion equalled observation.state[t+1] bit-exactly, which put 92.0% of its 6,075,103 rows outside the Box(low=-1, high=1, shape=(14,), dtype=float32) the environment declares. No rollout failure and no benchmark change was measured: it is a contract mismatch.

Units are the other half, and normalisation hides them. A LIBERO action of 1.0 is 5 cm or 0.5 rad for that control step, not a metre and not a radian; LeRobot joint targets are degrees when use_degrees=True, and its EEReferenceAndDelta step scales translation by end_effector_step_sizes but not rotation. The storage layer carries the same ambiguity, covered in the comparison of LeRobotDataset, RLDS and HDF5: joint targets, joint velocities, end-effector deltas and absolute poses are all called action on disk.

This week, run that check on one episode of your own data and write the answer into your collection contract as four fields: space, anchor, frame and units. Format conversion is scriptable any time. Ambiguous action-space semantics, once the operators who recorded the episodes have moved on, are not.