vLLM Multi-LoRA Serving: Adapter Limits and Silent Failures

Two rows need more than a cell.

--max-loras is a batching limit, not a hosting limit. In the v1 scheduler, the running set of adapter ids is tracked per step. A waiting request is skipped when the batch already holds max_loras distinct adapters and its adapter is not one of them; the code comment is Scheduling would exceed max_loras, skip. Base-model requests carry no adapter and are not counted. So with the default of 1, a claims request and an underwriting request never share a batch: one waits for the other's batch to drain. You can register far more adapters than --max-loras; it only bounds how many run together.

--max-cpu-loras defaults to --max-loras, not to unlimited. The config validator sets it equal to max_loras when unset, and rejects a smaller value with max_cpu_loras (X) must be >= max_loras (Y). With both at 1, the host cache holds one adapter, so alternating between two adapters reloads one from its path on every switch. The cache is an in-memory LRU: an evicted adapter is dropped and reloaded from lora_path on its next request, with no disk tier in between. It is unrelated to offloading base-model weights.

--specialize-active-lora only takes effect while cudagraph_specialize_lora is on, which is its default; the docstring trades better performance under variable adapter mixes for more startup time and memory.

For static serving, --lora-modules takes either name=path or a JSON object with name, path and base_model_name. This is the configuration we recommend for a regulated deployment: adapters from versioned directories, capacity set from what is actually served, runtime loading absent.

# first boot only: DEBUG surfaces 'No LoRA weights were applied for adapter ...'
export VLLM_LOGGING_LEVEL=DEBUG

vllm serve /models/Qwen3-8B \
  --enable-lora \
  --max-loras 4 \
  --max-lora-rank 64 \
  --max-cpu-loras 6 \
  --api-key "$VLLM_API_KEY" \
  --lora-modules \
    '{"name": "claims-triage", "path": "/adapters/claims-triage/v7", "base_model_name": "/models/Qwen3-8B"}' \
    '{"name": "underwriting-notes", "path": "/adapters/underwriting-notes/v3", "base_model_name": "/models/Qwen3-8B"}' \
    '{"name": "compliance-review", "path": "/adapters/compliance-review/v12", "base_model_name": "/models/Qwen3-8B"}'

--max-lora-rank 64 assumes 64 is the largest rank among these adapters. Drop the VLLM_LOGGING_LEVEL=DEBUG line after the first boot; it exists to catch one log message described below. Clients pick an adapter through the model field, and GET /v1/models lists each adapter with root set to its path and parent set to the base model.

BaseLinearLayerWithLoRA.create_lora_weights allocates, for every slice of every wrapped linear layer, a zero-filled lora_a of shape (max_loras, 1, max_lora_rank, input_size) and lora_b of shape (max_loras, 1, output_size, max_lora_rank), in lora_dtype, on the GPU. The buffers exist from startup, whether zero adapters are loaded or ten.

That makes the cost a product you can compute. On Qwen3-8B dimensions (hidden size 4096, intermediate size 12288, 36 layers, 32 query heads and 8 KV heads at head dimension 128, bf16), the A plus B elements per layer per unit of rank are 18,432 for the merged QKV projection, 8,192 for o_proj, 32,768 for the merged gate and up projection and 16,384 for down_proj: 75,776 in total. Across 36 layers at 2 bytes, that is about 5.5 MB per unit of rank per slot.

These are decimal gigabytes from the buffer shapes, assuming tensor parallel size 1, --fully-sharded-loras off, adapters on every attention and MLP projection and none on the embeddings or LM head. They are not observed memory, and they exclude any extra CUDA graph memory from --specialize-active-lora. Confirm against the startup memory log on your own card. What the table does show reliably is the shape: cost is linear in slots times rank, so 16 slots at rank 64 costs the same as 8 at rank 128.

The rank lever is the cheap one. vLLM's docs say to set --max-lora-rank to the largest rank you serve: for adapters at 16, 32 and 64, use 64 rather than 256, because a value much larger than needed wastes memory and can cause performance issues. On the dimensions above, 4 slots at rank 256 would preallocate about 5.6 GB to serve adapters that never exceed 64.

The memory comes out of the same GPU budget as the KV cache, so on a card that was already tight, adding slots shrinks how many tokens you can hold; the vLLM KV cache OOM debugging guide covers where that budget goes.

v0.32.0 also closed a crash in this area. Before PR 59801, an adapter with r=8 and a rank_pattern setting q_proj to 32 passed validation under --max-lora-rank 16 and then killed the engine with a tensor size mismatch. The validator now checks the largest rank in rank_pattern against max_lora_rank at load.

A multi-LoRA server reports success when an adapter registers. Registration does not prove the adapter's weights touch a single layer, or that they are scaled the way your training run scaled them. Every row below describes an adapter that registered without an error on at least one recent release. At v0.32.0 two of them no longer do: adapters that use an unsupported PEFT feature or carry bias terms fail the PEFT config check (validate_legal) or the tensor-name parser when the worker loads them, so they surface as a load error instead of as wrong output. Those two rows stay in the table for anyone auditing what an earlier release served.

The first row is the one to worry about because nothing above DEBUG reports it. PR 55310 deliberately made the message debug-level, because pipeline or expert parallelism can legitimately leave a worker with no local LoRA weights. At v0.32.0 the line reads: No LoRA weights were applied for adapter %s on this worker. Requests may use the base model; check --lora-target-modules. This may be expected with pipeline or expert parallelism. The default VLLM_LOGGING_LEVEL is INFO, and the worker manager still does not consult target_modules during validation.

Upstream already guards some of these cases. One model family's config check at v0.32.0 refuses to start when a fused collective path would bypass LoRA-wrapped layers and silently drop their LoRA. The general case still needs your own check.

The check is a probe: greedy-decode a fixed prompt set against the base model name and each adapter name, and flag any adapter that is identical to the base on every prompt. It uses only the documented /v1/completions fields and jq.

BASE=/models/Qwen3-8B
URL=http://localhost:8000/v1/completions
AUTH="Authorization: Bearer $VLLM_API_KEY"

complete() {  # $1 = model name, $2 = prompt
  jq -n --arg m "$1" --arg p "$2" \
     '{model: $m, prompt: $p, max_tokens: 64, temperature: 0}' |
  curl -s "$URL" -H "$AUTH" -H 'Content-Type: application/json' -d @- |
  jq -r '.choices[0].text'
}

for adapter in claims-triage underwriting-notes compliance-review; do
  same=0; total=0
  while IFS= read -r prompt; do
    total=$((total + 1))
    [ "$(complete "$BASE" "$prompt")" = "$(complete "$adapter" "$prompt")" ] && same=$((same + 1))
  done < probe_prompts.txt
  echo "$adapter: identical to base on $same of $total prompts"
done

Read the result asymmetrically. Identical on every prompt means the adapter is not applied, or is not active on these prompts; use prompts from the adapter's own domain. Different output proves only that something changed, not that the right adapter is loaded, so pair it with the root and parent fields from /v1/models. And because greedy output on the LoRA path can vary between runs (more on that below), run each prompt twice before trusting a differs verdict. Where adapters meet tool calling, the parser side has its own failure modes, covered in empty tool_calls and missing reasoning content in vLLM.

Smaller LoRA fixes shipped in the releases before that. v0.29.0 fixed an int32 overflow in the punica kernels at long context, false target matches on unsupported module types, and partial LoRA on Qwen3.5 and Qwen3.6 GatedDeltaNet layers. v0.30.0 fixed rsLoRA scaling in MoE expert packing and --lora-modules name=path with an = inside the path. v0.32.0 removed Tensorizer support from LoRA loading (PR 60024), so any loader built on it breaks on upgrade. Pin the version, and rerun the probe on every upgrade.

Setting VLLM_ALLOW_RUNTIME_LORA_UPDATING (parsed as true only for 1 or true) registers POST /v1/load_lora_adapter, which takes lora_name and lora_path plus optional load_inplace and is_3d_lora_weight, and POST /v1/unload_lora_adapter, which takes lora_name. When the variable is unset the routes are not registered at all.

vLLM is unambiguous about it. The LoRA doc says: "This feature comes with security risks. It should not be used in production unless it is an isolated, fully trusted environment." The server logs this at startup when the variable is on: LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! It also refuses to start with more than one API server process: VLLM_ALLOW_RUNTIME_LORA_UPDATING cannot be used with api_server_count > 1.

The authentication picture is the part to read closely. --api-key guards the /v1, /v2, /inference and /cohere prefixes, and the security doc lists the two /v1 LoRA endpoints as protected because they can alter model behaviour. At v0.32.0, though, the same load and unload handlers are also registered through the SageMaker hosting-standards library vLLM bootstraps at startup, which mounts POST /adapters and DELETE /adapters/{adapter_name}. /adapters is not under any guarded prefix, and vLLM's own test suite loads an adapter through it. That is a reading of the v0.32.0 source rather than a published advisory, and its consequence is simple: with runtime updating on, an API key does not stand between the network and a change to what a model name serves. The broader argument for authenticating at a proxy rather than in vLLM is in our vLLM inference server hardening guide.

Resolvers widen the same path. Both built-in LoRA resolvers require runtime updating. lora_filesystem_resolver, enabled through VLLM_PLUGINS with VLLM_LORA_RESOLVER_CACHE_DIR, loads directory foobar from that cache directory when a request names adapter foobar; write access to that directory becomes the ability to change what a model name serves. lora_hf_hub_resolver, driven by VLLM_LORA_RESOLVER_HF_REPO_LIST, downloads from the Hugging Face Hub, which the docs call insecure and not intended for production. Inside a closed perimeter it has no place; the offline posture is covered in air-gapped vLLM deployment.

Unload is weaker than its name. At v0.32.0 unload_lora_adapter deletes the name from the API server's registry and never calls the engine's remove_lora, so the weights stay in the worker's LRU until capacity evicts them. With a resolver configured and the directory still present, the next request for that name loads it again.

Our recommendation for a regulated deployment: static --lora-modules from a read-only, change-controlled path, runtime updating unset in production, and a proxy that refuses the mutating paths regardless, so that a restart with the wrong environment cannot open them.

# in the nginx server block that fronts vLLM
location ^~ /adapters                { return 403; }
location = /v1/load_lora_adapter    { return 403; }
location = /v1/unload_lora_adapter  { return 403; }

If you genuinely need hot loading, make it an admin-only route on a separate listener, which is what the security doc asks for: restrict these endpoints to trusted administrators with a reverse proxy or network-level controls.

Three v0.31.0 changes (5 October 2026) matter for multi-adapter serving. How block hashing works is covered in vLLM prefix cache misses on multi-turn agents; this is only what changed.

An adapter named after a served base model is now rejected. Before PR 59286, such an adapter was consulted before base models and silently captured every request for that model name. At v0.32.0 a runtime load returns 400 with The lora adapter '<name>' conflicts with a served base model., and a static --lora-modules entry with that name fails at startup.

A LoRA name and a cache_salt can no longer collide. Before PR 51899, a request for adapter foo and a base-model request with cache_salt set to foo produced the same first-block hash for the same prompt, so either could be served KV computed with the other's weights, and a caller could pick a salt that landed in an adapter's cache namespace. The fix shipped under advisory GHSA-rh6f-3x46-j33q (medium, affected below 0.31.0). The upgrade invalidates persisted LoRA and salted blocks once, because their hashes change.

The adapter path is now part of the block hash: the LoRA extra key is ('lora', lora_name, lora_path) (PR 59335). Re-pointing a name at a new path no longer reuses KV computed with the old adapter. An in-place reload from the same path with changed files is still not detected; vLLM only logs that prefix-cache blocks computed with the previous weights are not invalidated and that you should load changed weights from a new path. Content-hash versioning is an open PR.

The rule follows directly: version adapters by path (/adapters/claims-triage/v7, then v8), and never overwrite files in place.

Issue 56769, opened 14 September 2026, reports greedy decoding (temperature 0, fixed seed) that is deterministic on the base model and not through a rank-64 adapter targeting linear_attn modules, on Qwen3.8-27B-FP8. The reporter measured roughly 70 percent self-agreement for the adapter. A vLLM contributor replied that there are known sources of non-determinism in the LoRA operator path, and that PR 50278, which addresses one of them, is unmerged because it may introduce a performance regression.

v0.32.0 adds one LoRA-specific piece under VLLM_BATCH_INVARIANT=1 (PR 59377). The LoRA shrink kernel previously ran with SPLIT_K=1 in batch-invariant mode, because split-K normally reduces with float atomics. It now stores FP32 partials for 8 fixed splits and adds them in a fixed order. Batch-invariant LoRA output stays invariant across batch composition, but changes once relative to the earlier path because the reduction order differs, so re-baseline any golden outputs when you upgrade. This does not close 56769: the reporter could not enable batch invariance on that hybrid model at all, Qwen3.8 is not on the batch-invariance tested-models list, and the feature is documented as beta. General background on why temperature 0 is not deterministic is in why temperature 0 gives different results.

Issue 52568, opened 17 August 2026, is worse. On Qwen3.5-9B (a hybrid with 24 linear_attn and 8 self_attn layers) with a rank-64 adapter covering attention, linear attention and MLP modules, on an NVIDIA H20 at --max-loras 6, prompts above roughly 400 tokens return nothing but ! tokens: NaN logits, argmax to token 0. Short prompts are fine, and so are the base model, the merged model in vLLM and the adapter under Hugging Face PEFT inference. A commenter reported it still reproducing on 0.28.0. A contributor posted a proposed root cause and patch in the thread, but no fix had merged by this post's date.

Both reports are scoped to hybrid linear-attention bases with adapters on linear_attn modules. If that is your setup, put prompts well past 400 tokens into the probe set, and if they fail, serve the merged weights for that adapter instead of the LoRA path.

On fixed GPUs, one base model plus per-department adapters is the economical shape: one copy of the weights, one KV budget, one upgrade cadence. It is also where several fine-tuned specialists rather than one large model can share one set of base weights. The cost is that the adapter becomes the unit of change control. Even where a generator falls outside the formal model definition, as discussed in SR 26-2 model risk management scope for LLMs, the change log treats new weights as a change event, and an adapter is new weights. A reviewer will want evidence for each control below.

The last row is the honest one. On a dense model with batch invariance supported, you can state reproducibility; on a hybrid base with adapters on linear-attention modules, you document the open issues and the variance you accept. Batched multi-adapter serving itself is well studied (S-LoRA, arXiv 2311.03285, and Punica, arXiv 2310.18547), but those papers address throughput, not whether a regulator can trace a response to a validated adapter. More on model choice and serving across the cluster is in the LLM models pillar.

Three small checks. Boot your server once with VLLM_LOGGING_LEVEL=DEBUG and grep the log for No LoRA weights were applied. Run the base-versus-adapter greedy probe for every adapter you serve, with prompts from that adapter's domain. Then open the production unit file and confirm VLLM_ALLOW_RUNTIME_LORA_UPDATING is not set. If any adapter comes back identical to the base on every prompt, you have been serving the base model under that adapter's name.