Claude Code Local Model: What Actually Breaks in 2026

The reference settles the boring questions. The endpoints are /v1/messages and, optionally, /v1/messages/count_tokens, selected by ANTHROPIC_BASE_URL. The gateway must forward anthropic-beta and anthropic-version unchanged, and anthropic-version is currently 2023-06-01. Claude Code sends x-claude-code-session-id on every request and adds x-claude-code-agent-id and x-claude-code-parent-agent-id on subagent traffic; the agent IDs identify an agent, not a person or a device.

Capabilities that add body fields pair them with a beta header, and the pair travels together. A gateway that strips the header while passing the body, or forwards an Anthropic-format body to an upstream with a different schema, produces hard 400 errors, and only when both halves are absent together does the feature turn off quietly.

Four request shapes and one prompt mutation are what a home-grown endpoint has never had to handle.

The prompt mutation is the attribution block Claude Code prepends to the system prompt. api.anthropic.com strips it before processing, but the strip is positional: it fires only when the block arrives unchanged as the first system block, so prepending another block, reordering the array, or flattening it to a string defeats it. Any other upstream receives the block as part of the prompt. Since client v2.1.181 (npm, 2026-06-17) the block is stable for the lifetime of a conversation behind a custom base URL, which retires the older advice to always set CLAUDE_CODE_ATTRIBUTION_HEADER=0, and vLLM removes it anyway: serving.py at v0.28.0 skips any system text block starting with x-anthropic-billing-header.

One more behaviour is not a request at all. Claude Code counts every relayed byte on ANTHROPIC_BASE_URL connections, including SSE ping events and comment lines, and aborts a stream that goes silent for 300 seconds by default; CLAUDE_BYTE_STREAM_IDLE_TIMEOUT_MS moves that watchdog alone, clamped between 10 seconds and 30 minutes. A large model on an undersized box that prefills for six minutes without emitting anything hits the ceiling before producing a token, and vLLM's Anthropic streaming path at v0.28.0 has no keep-alive emission site: AnthropicStreamEvent declares ping in its type literal, but nothing in serving.py or api_router.py writes one.

Read at vLLM 0.28.0 (2026-08-26), Ollama 0.33.0 (2026-08-21, with 0.33.2 the newest patch before this post) and LiteLLM 1.98.0 (2026-08-23), the three backends behave identically on three rows: all serve POST /v1/messages, all serve GET /v1/models, and none honours cache_control. The other nine split.

The rows do not sort the way the reputations do. vLLM covers the most fields that reach the model and says the least when it drops one. Its AnthropicMessagesRequest model in vllm/entrypoints/anthropic/protocol.py at v0.28.0 declares thirteen Anthropic fields and no others: model, messages, max_tokens, metadata, output_config, stop_sequences, stream, system, temperature, tool_choice, tools, top_k and top_p, plus four vLLM-only fields including cache_salt. No request model in that file declares thinking or context_management, and cache_control appears nowhere in it at all. The router is not behind a flag: vllm/entrypoints/generate/api_router.py imports its attach_router and calls it inside register_generate_api_routers, which runs whenever generate is among the server's supported tasks.

Ollama publishes its gaps. Its Anthropic-compatibility page lists eight features as not supported: /v1/messages/count_tokens, tool_choice, metadata, prompt caching via cache_control blocks, the Batches API, Citations content blocks, PDF document content blocks, and server-sent errors during streaming. A separate behaviour-differences heading states three more: the API key is accepted but not validated, anthropic-version is accepted but not used, and token counts are approximations based on the model's tokenizer. Those are two distinct facts about tokens: the endpoint is absent, and the counts that do come back are estimates.

LiteLLM's /v1/messages is registered in litellm/proxy/anthropic_endpoints/endpoints.py at v1.98.0 under a tag that reads beta, and the handler's own docstring points operators at the {PROXY_BASE_URL}/anthropic/v1/messages passthrough route instead. It also adds a non-spec usage.total_tokens field by default, behind a litellm.strip_anthropic_total_tokens flag left off for backward compatibility, while the streaming SSE path already omits it. The same proxy returns two usage shapes depending on whether you streamed.

Claude Code has a real recovery path, and it is narrow. When the upstream rejects the thinking field, a thinking signature, a mid-conversation system message, or the cache_control marker on one of those messages, Claude Code retries and disables the rejected capability for the rest of the conversation. It does not retry context management or tool schema field rejections; those 400 errors reach the developer. The retry matches on the upstream's error wording, so a gateway that wraps errors in its own envelope breaks recovery unless the envelope carries a stable capability_rejected token.

Read that mechanism and the ranking inverts. A backend that rejects a field it cannot honour gives Claude Code a signal it can act on; one that accepts and ignores it returns a 200 indistinguishable from success.

vLLM at v0.28.0 is in the second category by construction. None of the Pydantic models in its Anthropic protocol file sets model_config or an extra policy; a grep for model_config, extra= and ConfigDict across that file and serving.py returns nothing. Pydantic v2 defaults to extra='ignore', so an unknown top-level field is dropped without an error. Send thinking to that router and you get HTTP 200, a normal message, a normal usage object, and no thinking. Nothing logs it on either side, and the retry path never fires because there was no error to trigger it.

This is the same class of problem as an endpoint that accepts a tool_use block with no matching tool_result, where your own endpoint does not enforce the pairing the hosted API enforces. The hosted API does validation work you inherit the moment you move the endpoint.

Six issues on the Claude Code repository describe base-URL behaviour, all open with a null closed_at when checked for this post. Three of the symptoms quoted from them need adjusting before they reach a runbook: one is documented intended behaviour, one was clamped client-side, and one contradicts the documented default.

#89211 contradicts the documented default. The report shows API Error: 400 Deferred custom tools are only supported on Anthropic models and on Anthropic-compatible provider endpoints that implement deferral, on client 2.1.241, with ENABLE_TOOL_SEARCH=false as the workaround. The environment variable reference says the opposite is current behaviour: when ANTHROPIC_BASE_URL points at a non-first-party host, MCP tool search is disabled by default and all MCP tools load upfront, and ENABLE_TOOL_SEARCH=true is the opt-in. Set the value explicitly so neither the default nor the report's premise decides it for you.

#82693's error is stale and its real defect is not. The API error string quoted in that report, about output_config.effort xhigh and disabled thinking, is what Claude Code showed before v2.1.242 (2026-08-24), and since v2.1.251 (2026-08-28) it sends effort high to models it knows reject the combination. The defect underneath is parameter inheritance, and it is untouched. With no custom base URL, the native WebSearch side query went out pinned to claude-haiku-4-5-20251001, max_tokens 32000, temperature 1 and no output_config. With ANTHROPIC_BASE_URL set, the same query inherited the session's values for all four instead: model claude-opus-5, max_tokens 64000, the session temperature, and output_config at effort xhigh. Only the effort level ever raised an error. The other three just run, at your session's cost.

#87876 reports three symptoms with three different statuses. Remote Control being unavailable behind a non-Anthropic base URL has been intended since v2.1.196 (2026-06-29), matching Bedrock, Google Cloud's Agent Platform and Microsoft Foundry. The 1M window dropping to 200k the report assigns to a separate issue, with the [1m] model ID as its workaround. The null usage meters are the part with evidence: the reporter logged Anthropic-Ratelimit-Unified-5h-Utilization: 0.09 and Anthropic-Ratelimit-Unified-5h-Status: allowed arriving on the same requests whose usage Claude Code then declined to display, through a pass-through proxy terminating at Anthropic. Setting the variable to the literal string https://api.anthropic.com keeps the meters working, and so does reaching the same proxy by DNS redirection with the variable unset. That is the whole post in one report: the client infers third-party provider status from the base URL string, not from the wire.

The numbers in #87227 are one reporter's, from session timings and their own vLLM engine counters across 2.1.232, published 2026-08-13, and 2.1.233 the next day. Pin the version; why a vLLM prefix cache stops hitting between turns is a separate investigation.

Claude Code sizes its compaction trigger from the model ID string. Behind a gateway that string is whatever you named the served model, so an alias picked to satisfy the model picker decides when conversations compact. CLAUDE_CODE_MAX_CONTEXT_TOKENS corrects it under three documented rules.

Now put rule 3 next to the discovery filter. Claude Code keeps a discovered model only when its id contains claude or anthropic anywhere in the string, case-insensitively; before v2.1.223 (2026-08-05) it had to begin with one of those. Ollama's published remedy is a rename, ollama cp qwen3-coder claude-3-5-sonnet. That fixes discovery and lands you in rule 3, where the window override needs DISABLE_COMPACT and therefore no compaction at all.

The way out is to satisfy the substring test without the prefix test: name the served model something that contains claude but does not start with claude-, for example internal-claude-coder, via --served-model-name on vLLM or ollama cp on Ollama. vLLM's Claude Code page adds a constraint, that model names cannot contain /, so a Hugging Face repo id is not a usable served name.

Ollama's launcher shows what happens when nobody chooses: cmd/launch/claude.go at v0.33.0 sets CLAUDE_CODE_AUTO_COMPACT_WINDOW only when the model name is recognised as a cloud model, so a locally pulled model gets no window declaration at all.

Two adjacent levers have limits. CLAUDE_CODE_AUTO_COMPACT_WINDOW is clamped to at least 100,000 tokens and at most the model's window, so it cannot match a gateway limit below 100,000. CLAUDE_CODE_DISABLE_UNKNOWN_MODEL_WINDOW_ENFORCEMENT=1 compacts only after the API rejects the conversation with a too-long error Claude Code recognises, which is useless if the gateway rewrites errors in its own wording.

The pre-flight below is nine requests, one per capability, and it belongs in the repository next to the deployment because it has to run on every version bump of either side. Set ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN and MODEL first. Checks 1 and 2 cover reachability and whether the token-counting endpoint exists.

# 1. reachability and credential
curl -sS -X POST "$ANTHROPIC_BASE_URL/v1/messages" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "'"$MODEL"'", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

# 2. is /v1/messages/count_tokens served, or does Claude Code fall back to inference?
curl -sS -o /dev/null -w '%{http_code}\n' -X POST "$ANTHROPIC_BASE_URL/v1/messages/count_tokens" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "'"$MODEL"'", "messages": [{"role": "user", "content": "Hello"}]}'

A response to check 1 starting with an id field proves reachability; an unknown-model error still proves the URL and the credential. A 404 on check 2 is expected on Ollama, not a bug: it tells you every context measurement will be spent as an inference request. Check 3 is what the /model picker would see; the grep is the test, not decoration.

# 3. what the /model picker would actually see
curl -sS --max-time 3 \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "x-api-key: $ANTHROPIC_AUTH_TOKEN" \
  "$ANTHROPIC_BASE_URL/v1/models?limit=1000" \
  | grep -iE '"id"[^,]*(claude|anthropic)' || echo 'FAIL: no discoverable id contains claude or anthropic'

Checks 4 to 7 send the four body fields that separate the backends, one at a time, because a combined request cannot tell you which field caused a 400.

probe () {  # $1 = label, $2 = extra JSON fragment
  code=$(curl -sS -o /tmp/probe.json -w '%{http_code}' -X POST "$ANTHROPIC_BASE_URL/v1/messages" \
    -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
    -H "anthropic-version: 2023-06-01" \
    -H "content-type: application/json" \
    -d "{\"model\": \"$MODEL\", \"max_tokens\": 16, $2, \"messages\": [{\"role\": \"user\", \"content\": \"hi\"}]}")
  printf '%-16s %s\n' "$1" "$code"
}

# 4. cache_control on a system block
probe cache_control '"system": [{"type": "text", "text": "You are a build agent.", "cache_control": {"type": "ephemeral"}}]'
# 5. output_config effort
probe output_config '"output_config": {"effort": "high"}'
# 6. extended thinking
probe thinking '"thinking": {"type": "enabled", "budget_tokens": 1024}'
# 7. tool forcing
probe tool_choice '"tool_choice": {"type": "any"}, "tools": [{"name": "noop", "description": "does nothing", "input_schema": {"type": "object", "properties": {}}}]'

Two hundred is not a pass on checks 4 through 6. For cache_control, usage in /tmp/probe.json is weaker evidence than it looks: vLLM fills cache_read_input_tokens from its own prefix cache whether or not the marker survived, so a populated counter proves engine-side caching, not that the field was read. The request schema is the answer there. For output_config, only vLLM declares the field, so a 200 from the other two leaves the question open. For thinking against vLLM 0.28.0 the field is not in the schema, Pydantic drops it, and the status is 200 with the capability gone. Check 8 asks whether anything reaches the client during a long prefill; ts comes from moreutils, and any timestamping filter does the job.

# 8. does the endpoint keep the wire warm while it prefills?
curl -sS -N -X POST "$ANTHROPIC_BASE_URL/v1/messages" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "'"$MODEL"'", "max_tokens": 64, "stream": true, "messages": [{"role": "user", "content": "'"$(head -c 200000 /dev/urandom | base64 | tr -d '\n' | head -c 120000)"'"}]}' \
  | ts '%.s' | head -40

The timestamps on the first lines are the result, not the completion: if the first byte arrives more than 300 seconds in, the byte watchdog aborts the stream before the model produces a token. Check 9 is a smoke test for prompt reshaping. It does not prove the attribution strip works, because a server that merges the array into one string still answers correctly; its value is catching a proxy that drops or reorders entries.

# 9. does the endpoint preserve a two-entry system array, block order intact?
curl -sS -X POST "$ANTHROPIC_BASE_URL/v1/messages" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "'"$MODEL"'", "max_tokens": 64, "system": [{"type": "text", "text": "BLOCK_ONE_SENTINEL"}, {"type": "text", "text": "Repeat the first word of your system prompt and nothing else."}], "messages": [{"role": "user", "content": "go"}]}'

Assume the decision to terminate inference in-boundary is settled, in the case for terminating inference inside the enclave. Three things change the moment the base URL stops being api.anthropic.com, and a security review will ask about each.

First, capability loss is invisible, which is an audit problem before an engineering one: on a server whose request model ignores unknown fields, an audit trail built on status codes proves nothing about what the model received.

Second, the client keeps talking to hosts outside the enclave even when inference does not. CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns off auto-updates, telemetry, error reporting, release notes, availability checks and gateway model discovery refreshes, so the one variable that satisfies an egress rule is also the one that stops the picker's gateway list refreshing, and setting it to 0 or false still disables the traffic: only unsetting it restores any. The WebFetch domain safety check separately calls api.anthropic.com unless skipWebFetchPreflight: true is set, and telemetry still leaves for Anthropic while a gateway is configured. It goes without the gateway credential, but only since v2.1.246 (2026-08-25): earlier clients could attach it. An air-gap claim has to enumerate those paths rather than assume the base URL covered them.

Third, one boundary control misfires on this client specifically. A WAF in front of the endpoint returns a 403 with an HTML body while the gateway logs show no request at all, because coding-agent prompts contain XML-style tags and source code matching cross-site-scripting body rules, so a curl smoke test passes and a real session does not. Anthropic names the fix: exempt the gateway's /v1/messages path from request-body inspection, with the AWS WAF CrossSiteScripting_Body managed rule as the example. The controls are fine. The agent's traffic does not look like what they were written for.

Three placements for the Anthropic format exist, and all three work at these versions.

Which one you pick is a question about policy ownership, not a ranking: per-user attribution and quota need a proxy, and picking the gateway product itself is a separate decision, while the widest field coverage with fewest hops is vLLM's native router. Our AI development tools guides cover the adjacent pieces: when a vLLM tool parser returns an empty tool-call array and confirming whether your prompt cache is actually being read.

What the runbook pins:

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://llm.internal:8000",
    "ANTHROPIC_AUTH_TOKEN": "replaced-by-apiKeyHelper-in-production",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "internal-claude-coder",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "internal-claude-coder",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "internal-claude-coder",
    "CLAUDE_CODE_SUBAGENT_MODEL": "internal-claude-coder",
    "CLAUDE_CODE_MAX_CONTEXT_TOKENS": "131072",
    "ENABLE_TOOL_SEARCH": "false",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "DISABLE_AUTOUPDATER": "1"
  }
}

The served name is internal-claude-coder because it clears the discovery substring filter without starting with claude- or naming a Claude model, which keeps CLAUDE_CODE_MAX_CONTEXT_TOKENS in the first of the three rules. All four model variables are pinned explicitly, which is what makes CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC safe here despite it turning off discovery refreshes. DISABLE_AUTOUPDATER is there because a client auto-update is a version change to one half of a two-sided protocol: the version table in #76295 runs from 2.1.91 working to 2.1.206 still broken. In production, replace the literal token with an apiKeyHelper, whose output is cached for five minutes by default (CLAUDE_CODE_API_KEY_HELPER_TTL_MS).

This week, run the nine checks and paste the nine results into the runbook beside the client and server versions you tested them on. If check 4, 5 or 6 returns 200 and the capability is still missing from the response, you have found the failure this post is about, before someone spends a day wondering why the model stopped thinking.