Unsloth vs Axolotl vs LLaMA-Factory vs TRL: How to Pick

TorchTune appears in that speed table, in the roundups that copy it, and in a surprising number of 2026 shortlists. It should not appear in any of them.

The torchtune README carries a banner reading, verbatim, "Torchtune is no longer actively maintained: torchtune development wound down in 2025". Issue 2883 in the repository, titled "The future of torchtune" and opened 2025-07-15, states "We are stopping active development on torchtune, effective immediately" and "No new features will be added to the library", with critical bug fixes and security patches promised only through 2025. The last release on PyPI is 0.6.1, published 2025-04-07, which is sixteen months old as of this post. The project also moved repositories along the way, so older links redirect and give the impression of activity.

Recommending it to a regulated client in 2026 means recommending an unmaintained training dependency for a model that will end up in a system of record. The choice is Unsloth, Axolotl or LLaMA-Factory, plus TRL as the library the other three build on or around.

Release cadence is the sleeper column. TRL shipped 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.9.0, 1.9.1 and 1.9.2 between 2026-05-27 and 2026-07-28. LLaMA-Factory shipped 0.9.4 on 2025-12-31 and 0.9.5 on 2026-05-30, roughly two releases a year. Neither cadence is wrong, but they imply very different pinning strategies: TRL needs a lockfile and a weekly look at the changelog, LLaMA-Factory needs a security watch on the advisory feed.

Feature lists no longer separate these tools. All of them do LoRA, QLoRA and full fine-tuning; LLaMA-Factory's README lists pre-training, SFT, reward modeling, PPO, DPO, KTO, ORPO and SimPO, and TRL covers SFT, DPO and GRPO first-party. Reading a feature matrix in 2026 tells you almost nothing.

Four things diverge, and every one of them can invalidate a plan after you have booked GPU time:

Unsloth publishes per-step timings and peak VRAM on its own documentation, with hardware, model, context length and baseline stated. That is vendor-published and should be read as such, but it is measured, specific and falsifiable, which puts it in a different category from the four-number table above.

gpt-oss-20b BF16 on a B200, Unsloth against Transformers v5:

The obvious conclusion is that the advantage grows with context. That conclusion is wrong, and the counter-example is on the same page. Qwen3-30B-A3B on the same B200 at LoRA rank 8:

Same vendor, same page, same GPU, opposite direction. The speedup is a property of how well the kernels cover that specific architecture, not a property of context length. Anyone quoting one curve as the general behavior of the library has not read the second table.

Two more numbers deserve their caveats. Unsloth's MoE headline is "~12x faster Mixture of Experts (MoE) LLM training with >35% less VRAM", but no published table on that page reaches 12x; the highest measured entry is the 7.3x above. And the famous "30x faster" figure comes from the December 2023 launch post, where it describes two Tesla T4s under DDP going from 164 hours to 5 hours on a public dataset, using the closed Max tier. The open-source library was described on that same page as 2x faster with 50 percent less memory, and today's README says up to 2x faster with 70 percent less VRAM.

One useful independent signal: the Transformers v5 memory regression that makes the right-hand columns look so bad is not only an Unsloth talking point. Axolotl issue 3374, opened 2026-01-27, is titled "Transformers-v5 causes OOM on QLoRA config previously working on 4.57.6 (GLM-4.5-Air on 2x RTX Pro 6000)". Two projects with no shared interest observed the same thing.

For on-premise buyers this is the table that actually spends money, because it sets how many cards you order and which ones. Our RTX PRO 6000 against H100 against L40S comparison covers the card-level economics; this is the training-side demand that feeds into it.

Both projects call these minimums, and both are quoting short sequences. Context is what actually eats the card. Unsloth's own benchmark on Llama 3.1 8B QLoRA at rank 32, batch size 2 and gradient accumulation 4 shows maximum trainable context of 2,972 tokens on 8 GB where the Hugging Face plus Flash Attention 2 baseline goes out of memory, 21,848 against 932 on 12 GB, 40,724 against 2,551 on 16 GB, 78,475 against 5,789 on 24 GB and 342,733 against 28,454 on 80 GB.

MoE models sit outside these tables and need their own figures. Unsloth lists gpt-oss-20b at 14 GB for QLoRA on the model page and reports a 12.8 GB measured peak on the MoE kernel page, gpt-oss-120b at 65 GB QLoRA and 210 GB BF16 LoRA, and Qwen3-30B-A3B at 63 GB for 16-bit LoRA. The 12.8 and the 14 are not the same claim, so do not average them into one number for a capacity plan.

If you are still deciding whether to fine-tune at all rather than which tool to use, start with prompt engineering against fine-tuning and how much data a fine-tune actually needs. The framework question only becomes real once those two are settled.

Here is the correction that matters more than any wall-clock figure. "Unsloth is single-GPU only" is repeated everywhere, including in competing projects' comparison material. It is stale. Unsloth's multi-GPU documentation covers DDP and FSDP through Accelerate, DeepSpeed via Accelerate, accelerate launch train.py, torchrun --nproc_per_node N_GPUS train.py, and device_map = "balanced" for models that will not fit on one card.

The sharper and more useful statement is this: Unsloth does data parallelism only. Tensor, context and expert parallelism are not documented at all, and the multi-GPU path has open correctness bugs, including issue 3915 from 2026-01-21 ("Multi GPU (DDP) is not working in any setting"), issue 6831 from 2026-07-02 reporting silent multi-GPU corruption with a layer-40 reproduction, and issue 4142 from 2026-03-02 reporting multi-node broken specifically in the one-GPU-per-node configuration. The docs themselves note the process is complex and requires manual setup.

Two caveats keep this table honest. Axolotl's parallelism guide publishes FSDP plus TP, HSDP plus TP and FSDP plus TP plus CP as supported, but the config reference help string for tensor_parallel_size reads "Only supported with DeepSpeed AutoTP", which is materially narrower than PyTorch-native tensor parallelism composed with FSDP2. Architect against the config reference, not the guide. And expert_parallel_size appears in that guide but was not found in the config reference, so treat it as guide-documented and reference-unconfirmed until you see it accepted by your install.

# axolotl 0.18.0, MoE QLoRA with expert quantisation
base_model: zai-org/GLM-4.7-Flash
load_in_4bit: true
adapter: qlora
quantize_moe_experts: true    # requires lora/qlora with load_in_4bit or load_in_8bit

sequence_len: 32768
sample_packing: true
flash_attention: true
micro_batch_size: 1

fsdp_version: 2
fsdp_config:
  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
  fsdp_activation_checkpointing: true
  fsdp_reshard_after_forward: true

dp_shard_size: 4
dp_replicate_size: 1
tensor_parallel_size: 1       # DeepSpeed AutoTP only
context_parallel_size: 2      # divisor of GPU count
# sequence_parallel_degree: 4 # DEPRECATED, use context_parallel_size

That deprecated key is a live foot-gun. The most-cited Axolotl long-context benchmark, published 2025-04-04, uses sequence_parallel_degree throughout, and its YAML has been copied into 2026 posts unchanged. The benchmark's scaling shape is still worth knowing, on Llama 3.1 8B on H100:

Sequence parallelism buys context, not throughput. At degree 8 you are paying eight GPUs for 15.2 percent per-GPU efficiency. That is a rational trade when the alternative is not training at all, and an irrational one when a shorter context would have worked.

TRL splits the same problem across two backends with hard version floors, and picking the wrong one wastes a day:

from trl import SFTConfig

training_args = SFTConfig(
    pad_to_multiple_of=4,          # cp_size * 2
    max_length=16384,              # global length, TRL splits it
    packing=True,
    use_liger_kernel=True,
    gradient_checkpointing=False,  # cannot co-exist with fsdp_activation_checkpointing
    per_device_train_batch_size=1,
)

TRL's own documentation states that with 8 GPUs, context lengths over 300k tokens become feasible, and the page carries a banner marking the section as under construction. Plan for the API to move.

Unsloth's speed comes from architecture-specific kernels, and coverage is not an architecture check at runtime. It is a lookup table keyed by exact Hugging Face repository id. unsloth/models/mapper.py defines INT_TO_FLOAT_MAPPER, FLOAT_TO_INT_MAPPER, MAP_TO_UNSLOTH_16bit, FLOAT_TO_FP8_BLOCK_MAPPER and FLOAT_TO_FP8_ROW_MAPPER, with roughly 250 or more entries in the core dict, each one a literal string like unsloth/mistral-7b-bnb-4bit.

If your checkpoint is not in that table, you do not get a degraded version of the advantage. You get the generic path, and the entire reason you chose the library evaporates.

python - <<'PY'
import unsloth.models.mapper as m

tables = {k: getattr(m, k) for k in dir(m)
          if k.isupper() and isinstance(getattr(m, k), dict)}
print("mapper tables:", {k: len(v) for k, v in tables.items()})

CKPT = "meta-llama/Llama-3.1-8B-Instruct"   # your exact repo id
hits = {name: (CKPT in tbl or CKPT in tbl.values()) for name, tbl in tables.items()}
print(CKPT, "->", hits)
if not any(hits.values()):
    print("NOT MAPPED: expect the generic path, none of the published speedup.")
PY

The failure modes when coverage is missing are loud, at least. Verbatim from the issue tracker: "unsloth/gemma-4-E4B-it is not supported in your current Unsloth version!" (issue 4942, 2026-04-10), "unsloth/Qwen3.5-0.8B is not supported yet in transformers==4.57.6" (issue 4146, 2026-03-03), and "NYI: Currently non power of 2 embedding dimension are not supported" (issue 4036, 2026-02-11). Run the load once on a cheap instance before you reserve a week of cluster time. The same discipline applies to a base model you may not keep, which is the argument in large model against fine-tuned small model.

For a regulated deployment the run has to leave behind an artifact an auditor can read: base model and revision, dataset version, hyperparameters, seed, framework version, and the exact hardware. Axolotl wins this on structure alone, because the whole run is one YAML file you commit next to the code. TRL's Python configs are reviewable but easier to drift, and a notebook-driven Unsloth workflow is the hardest of the four to reconstruct six months later, which is a process problem rather than a library defect.

LLaMA-Factory earns its 73,845 stars on ergonomics: llamafactory-cli webui gives a non-specialist a working LoRA run in an afternoon, and use_unsloth: true puts Unsloth kernels underneath it. Note that both use_unsloth and use_unsloth_gc default to False in src/llamafactory/hparams/model_args.py, so the acceleration is opt-in and applies to LoRA training only. Anyone reporting LLaMA-Factory timings without publishing the config has not told you whether the kernels were on.

The tail on that convenience is a web UI that has repeatedly been reachable from the network:

Nothing below 0.9.4 belongs anywhere near regulated data, and the training UI should never be exposed beyond localhost or a bastion. No published advisories were found for unsloth, axolotl or trl at the time of writing, which is not the same as there being none.

pip install "unsloth==2026.8.5" "unsloth_zoo==2026.8.4"
pip install "axolotl==0.18.0"        # Python >=3.10
pip install "llamafactory>=0.9.4"    # Python >=3.11, never pin below 0.9.4
pip install "trl==1.9.2"             # Python >=3.10
# torchtune is not a live option: last release 0.6.1, 2025-04-07

Across the on-premise stacks we audit at Particula Tech, the recurring gap is not framework choice at all. It is that six months after a fine-tune ships, nobody can produce the dataset revision it was trained on. Pick the tool that makes that record fall out of the build, and treat the environment itself as part of the record: the same air-gapped discipline described in our offline vLLM deployment guide applies to training hosts, and the wider build-versus-buy tradeoff is in cloud against on-premise AI security and cost.

Concretely: use Unsloth on one GPU after you have grepped mapper.py for your checkpoint, and use Axolotl the moment you need a second node or a parallelism axis beyond data. Use TRL when you are writing the loss function yourself. Use LLaMA-Factory when a non-specialist team needs results this week, pinned at 0.9.4 or later, with the UI bound to localhost. Do not use torchtune. And do not quote the 3.2 hours against 5.8 hours table in a hardware proposal, because the first person who asks for the config will find out there isn't one.

The framework is the smallest decision in this stack anyway. Where the model runs afterward, and what evidence you kept about how it was built, are the ones that survive an audit. That larger picture is laid out in our LLM models pillar guide.