Transformers Does Not Recognize This Architecture in vLLM

vLLM startup produces seven messages that all read as "your model is not supported" and want different fixes. Two come from Transformers and fire on model_type. Four come from ModelRegistry._raise_for_unsupported in vllm/model_executor/models/registry.py and fire on architectures. One comes from resolve_trust_remote_code and is about vendor Python, not support at all.

The removed-architecture branch inverts the usual instinct. _PREVIOUSLY_SUPPORTED_MODELS holds 37 entries mapping an architecture to the last vLLM version that carried it: Phi3SmallForCausalLM at 0.9.2, JAISLMHeadModel at 0.22.0, QWenLMHeadModel at 0.23.0. Upgrading again moves you further from a working server.

One detail hides all of this. ModelConfig is a pydantic dataclass, declared with arbitrary_types_allowed at vllm/config/model.py lines 121 to 122, so the ValueError surfaces wrapped in a ValidationError: 1 validation error for ModelConfig header. Grep the log for the backtick-quoted model type, not for the pydantic header.

The ordering inside ModelConfig.__post_init__ at v0.27.1 decides which fixes are reachable:

The config parse strictly precedes architecture resolution. A model_type miss at 581 aborts 61 lines before vLLM's own model registry is asked a question, which is why every piece of advice built on architectures, model implementations or plugins is inert against this error.

Inside get_config, HFConfigParser.parse does something bare Transformers never does: it reads model_type from the raw config_dict, checks it against _CONFIG_REGISTRY in vllm/transformers_utils/config.py, and on a hit calls _register_config_class, a thin wrapper over AutoConfig.register(model_type, config_class, exist_ok=True). Only then does it call AutoConfig.from_pretrained. That registry is a LazyConfigDict(dict) with 64 entries at v0.27.1, mapping model types to config class names: chatglm to ChatGLMConfig, deepseek_v4 to DeepseekV4Config, kimi_k3 to KimiK3Config, ultravox to UltravoxConfig.

Two consequences follow. A checkpoint can load under vLLM and fail under bare Transformers of the identical version, because vLLM registered a config class Transformers does not ship. And the fix is sometimes a vLLM upgrade, the opposite of what the error text implies. Three model types skip the path entirely: eagle, speculators and medusa match _SPECULATIVE_DECODING_CONFIGS and instantiate their config class directly, never touching AutoConfig.

vLLM does try to print something friendlier. It wraps the AutoConfig call in except ValueError and rewrites the message only when the exception contains the literal string requires you to execute the configuration file. That string does not exist anywhere in the Transformers 5.15.1 source tree, so the hint suggesting trust_remote_code=True never fires on Transformers 5 and the raw error passes through untouched. Same category as the vLLM parser bug behind empty tool_calls with a populated reasoning field: code doing exactly what it says, against a string the other side stopped emitting.

Print the three config.json fields that decide which branch you are in:

python -c "import json,sys; c=json.load(open(sys.argv[1]+'/config.json')); \
print('model_type:', c.get('model_type')); \
print('architectures:', c.get('architectures')); \
print('auto_map:', c.get('auto_map'))" /models/my-checkpoint

Then test the model type against both registries and the architecture against vLLM's:

python -c "from transformers.models.auto.configuration_auto import CONFIG_MAPPING; print('qwen3_5' in CONFIG_MAPPING)"
python -c "from vllm.transformers_utils.config import _CONFIG_REGISTRY; print('qwen3_5' in _CONFIG_REGISTRY)"
python -c "from vllm import ModelRegistry; print(sorted(ModelRegistry.get_supported_archs()))"

Both tests are cheap: CONFIG_MAPPING.__contains__ checks the generated mapping and the extra content dict without importing a config class, and _CONFIG_REGISTRY subclasses dict.

The answers diverge. At the versions studied, qwen3_5 and qwen3_5_moe are present in both Transformers 5.15.1 and vLLM 0.27.1, while glm_ocr, glm4_moe_lite and gemma4 are present in Transformers only. Test against the wheel you deploy, not the docs for a version you do not have: CONFIG_MAPPING_NAMES is imported from a 1,292-line generated auto_mappings.py, and configuration_auto.py adds five non-inferrable entries by hand plus the deprecated gpt-sw3 key, so the runtime mapping is slightly larger than the file on disk. Same thirty seconds of checkpoint reading that saves a wasted allocation when sizing a large MoE checkpoint before serving it.

The error prints two commands. The first is correct more often than its reputation suggests; the second is a trap in most production environments.

pip install --upgrade transformers
pip install git+https://github.com/huggingface/transformers.git

The upgrade works when your installed Transformers predates the model type. It cannot work when vLLM's own requirement caps you below the release you need, the entire content of vLLM issue #39216 from 2026-04-07: a PyPI vLLM pinning transformers < 5 against a checkpoint whose config class only exists in 5.x.

Read the left column before running the upgrade. On 0.19.x the real fix is a vLLM upgrade; the Transformers upgrade is a no-op that leaves the same error and the impression the advice was wrong rather than misaddressed.

From v0.24.0 onward there is no upper bound, which cuts both ways. The upgrade is unconstrained above, so the same command that resolves a brand new open-weight model landing before the serving stack catches up can pull a Transformers release vLLM has never been tested against. Pin the exact version you validated.

This flag is a standard suggestion and almost never applies to this error. Transformers computes eligibility from the config dict alone, at configuration_auto.py lines 389 and 390 of v5.15.1:

trust_remote_code is consulted only when has_remote_code is true. On a checkpoint with no auto_map, the common shape for a first-party release from a major lab, the flag cannot reach the code path that raises the architecture error. Adding it changes nothing and costs a restart.

When auto_map.AutoConfig is present and the flag is absent you get a different message from resolve_trust_remote_code: the repository contains custom code which must be executed to correctly load the model, with an instruction to pass trust_remote_code=True. The checkpoint carries vendor Python, the vendor expects you to run it, and your decision is a security decision rather than a version decision.

vllm serve accepts both --trust-remote-code and --no-trust-remote-code, because vLLM assigns argparse.BooleanOptionalAction to every bool field on ModelConfig, and trust_remote_code defaults to False. One documented failure mode does not apply to vLLM users: with trust_remote_code left as None, the Python API default rather than vLLM's, Transformers prompts interactively on a 15 second timeout, and any exception from that input() call, which is what a container with no TTY produces, becomes the same hard ValueError. vLLM always passes an explicit boolean.

Documented in vLLM's source and almost nowhere else. HFConfigParser.parse reads hf_overrides before the registry lookup, under a source comment stating the intent outright: allow hf_overrides to override model_type before checking _CONFIG_REGISTRY. A model_type key in the override dict replaces the on-disk one for that lookup.

The branch that follows is the useful part. When the overridden model type is in _CONFIG_REGISTRY and differs from what config.json says, vLLM registers the same config class under both names, so AutoConfig.from_pretrained returns the right class whatever the checkpoint claims. It then sets trust_remote_code = False, on the comment that once registered it is not remote code anymore.

vllm serve /models/my-checkpoint \
  --hf-overrides '{"model_type": "deepseek_v4"}'

This lies to the loader, so the substitution has to be genuine. vLLM ships the pattern internally: _CONFIG_REGISTRY maps kimi_k2 to DeepseekV3Config with the in-source comment that Kimi K2 uses the same architecture as DeepSeek V3, and maps both RefinedWeb and RefinedWebModel to RWConfig for the older Falcon repositories. Use it when the checkpoint is an architectural clone under a new vendor name, never to force unrelated classes together.

Two limits. --hf-overrides JSON-parses its value only when it matches a leading and trailing brace, so pass a single quoted JSON object. And the sibling override people reach for first does not work here:

vllm serve /models/my-checkpoint \
  --hf-overrides '{"architectures": ["LlamaForCausalLM"]}'

Flat overrides are applied with config.update(hf_overrides_kw) after AutoConfig.from_pretrained has returned, and dict-valued ones later still, at model.py line 593. Neither can prevent a failure during the AutoConfig call. The architectures override has one legitimate use, which vLLM logs when it applies: a config.json with no top-level architectures field, where the server tells you it expects hf_overrides={'architectures': ['...']} in the engine args.

Treat the remap as documented behaviour of v0.27.1 rather than a stable public contract: a private registry and an internal ordering, pinned to a version you tested.

ModelImpl is a Literal["auto", "vllm", "transformers", "terratorch"] at vllm/config/model.py line 107, and the field is declared model_impl: str | ModelImpl = "auto" at line 339. Because str sits in that union, vLLM's argparse builder emits a metavar rather than choices, so the CLI will not reject an unknown string. Stay inside the four documented values.

vllm serve /models/my-checkpoint \
  --model-impl transformers \
  --trust-remote-code

This resolves architectures, not model types, at line 642. It cannot rescue a config parse that already threw at 581. Where it helps is the fourth triage row: the config parsed cleanly, the architecture class is not in vLLM's registry, and the model is implemented in Transformers.

The backend has entry requirements, all from vLLM's supported-models documentation at v0.27.1: config.json must expose auto_map.AutoModel; customisation belongs in the base model class rather than the causal LM wrapper; kwargs must be threaded down to the attention module; attention must be called through ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]; and the model class must set _supports_attention_backend = True. Scope covers embedding, language, vision-language and audio-language models, encoder-only, decoder-only and mixture-of-experts, with image inputs only and video listed as future work.

Diagnostics differ by how you asked. Under --model-impl transformers, the resolver names the architecture and says either that it is not a registered Transformers model and AutoModel is missing from auto_map, or that its Transformers implementation is not compatible with vLLM. Under the default auto it returns silently and you fall through to the generic unsupported-architecture error. Ask explicitly when debugging.

For a genuinely custom architecture, register out of tree through the vllm.general_plugins entry point rather than patching vLLM:

# The entrypoint of your plugin
def register():
    from vllm import ModelRegistry

    ModelRegistry.register_model(
        "YourModelForCausalLM",
        "your_code:YourModelForCausalLM",
    )

The string form matters. Passing the class directly imports your module in the parent process, producing RuntimeError: Cannot re-initialize CUDA in forked subprocess when that module touches CUDA at import; the lazy <module>:<class> string defers it, and register_model rejects any string not in that shape. None of this helps with an unrecognised model_type: register_model is keyed on the architecture class name and runs after config parsing.

Confirm what actually loaded:

from vllm import LLM
llm = LLM(model="/models/my-checkpoint")
llm.apply_model(lambda model: print(type(model)))

A printed type beginning with Transformers means the Transformers modeling backend is in use. The server also logs Resolved architecture: at startup, the fastest confirmation in a running deployment. Model coverage is a real selection criterion between serving stacks, and one axis in our SGLang and vLLM comparison.

The error message's own second remedy is pip install git+https://github.com/huggingface/transformers.git. Behind an egress deny that is a dead end by construction, so the first instinct fails before it starts. Three options survive, and they rank cleanly.

The Transformers bound stops being a hotfix and becomes a change-controlled artifact. Whoever runs the internal package mirror carries a specific release, and the vLLM version already deployed decides whether that release is installable at all: the < 5 cap on 0.19.x is the trap, the >= 5.5.3 floor from 0.24.0 is the shape you want. The surrounding network posture, including the full egress inventory vLLM touches at startup, is covered in our air-gapped vLLM deployment guide.

--trust-remote-code in a regulated environment is not a flag, it is a decision to execute vendor Python from the checkpoint directory inside the inference process. The saving grace is that auto_map enumerates precisely which files are involved, so the review is bounded: read them, diff them against the previous revision when the vendor updates the repository, and vendor the result into your own artifact store rather than resolving it at runtime. The threat model we argue separately in hardening a vLLM inference server.

The model_type remap survives the perimeter best. It needs no new wheel, no remote code and no network, because it makes vLLM register a config class it already ships under the name your checkpoint uses. When the substitution is genuine, it turns a procurement problem into a one-line engine argument.

One consequence holds whichever path you take. This class of failure is deterministic and reproducible without weights or a GPU: --load-format dummy initialises weights with random values, and the run still walks the config parse and the architecture registry first:

vllm serve /models/my-checkpoint --load-format dummy --max-model-len 2048

That belongs in the model admission checklist that runs when a checkpoint lands in the artifact store, not in the incident channel at 2am with a reserved node burning budget. Take the last three checkpoints you admitted, run the config.json print against each, then the two membership tests against the Transformers and vLLM versions you actually deploy. Any checkpoint missing model_type from both registries is a startup failure waiting for the next node reservation, and you now know whether the answer is a mirror ticket, a reviewed auto_map, or a one-line override. Once the server is up, the failures move to runtime, starting with prefix cache misses in multi-turn agent traffic.