vLLM Air Gapped Deployment: Block Hugging Face Calls

Deal with this first, because it will waste an afternoon otherwise. vLLM's own documentation uses "offline inference" to mean the in-process Python LLM class as opposed to the HTTP API server. The offline inference page says, in effect, that offline inference is possible in your own code using vLLM's LLM class. It is a statement about process topology, not about network isolation. Batch inference through LLM() will happily download a model from the Hub.

So searching vLLM's docs for "offline" lands you on the wrong page, and several tutorials currently ranking for air-gap queries cite that page as evidence of air-gapped support. It is not. The word that means what you mean is "air gapped", or more precisely "egress-denied". Use it in your own tickets and runbooks so the ambiguity does not propagate into your change request.

vLLM documents its model resolution order explicitly, and reading it is the fastest way to understand why a weights folder is insufficient (official docs, vLLM design documentation):

Steps 3 and 5 are separate network paths from step 6. That is the whole reason a weights directory is not a deployment. A local path short-circuits step 1 and usually step 6, because both read files by name. Steps 3 and 5 resolve through the transformers stack, and whether they hit the network depends on whether the resolver was handed a path or a repo id on that particular code branch for that particular architecture.

The practical consequence: always pass --tokenizer with the same local path rather than relying on it to be inferred, and always pre-populate the Hugging Face cache even when you have a local weights directory. The cache is what catches steps 3 and 5 when the path does not.

Here is the full inventory for a normal serving stack, with the switch and the default for each. Everything in the "Default" column is what you get if you change nothing.

Two rows deserve more than a table cell.

The telemetry call is real outbound HTTPS from a serving process. VLLM_USAGE_STATS_SERVER defaults to https://stats.vllm.ai (official docs). vLLM gives you four equivalent ways to kill it: VLLM_NO_USAGE_STATS=1, VLLM_DO_NOT_TRACK=1, the cross-vendor DO_NOT_TRACK=1, or the presence of the file ~/.config/vllm/do_not_track. The file-based switch is the one to prefer in a hardened image, because it survives someone shipping a new env file. The local statistics it would otherwise write land in ~/.config/vllm/usage_stats.json.

Per-request media fetch is inference-time egress, which no startup test catches. On a stock multimodal server, a user-supplied image URL in a chat request becomes an outbound HTTP call from your GPU node. The documented defaults: VLLM_IMAGE_FETCH_TIMEOUT 5 seconds, VLLM_VIDEO_FETCH_TIMEOUT 30 seconds, VLLM_AUDIO_FETCH_TIMEOUT 10 seconds, VLLM_MEDIA_FETCH_MAX_RETRIES 3, and VLLM_MEDIA_URL_ALLOW_REDIRECTS True. So one request is up to three attempts that follow redirects to wherever they point. Fetched media is cached with VLLM_MEDIA_CACHE_MAX_SIZE_MB at 5120 (5 GB) and VLLM_MEDIA_CACHE_TTL_HOURS at 24; setting VLLM_MEDIA_CACHE to an empty string disables the cache entirely.

This is a different failure mode from the startup downloads, and it is the one that matters for compliance. A server that passed a boot-time egress test can still make an outbound call six weeks later because a user pasted a URL. The alternative, --allowed-local-media-path, lets requests reference files on the server filesystem instead, and vLLM's own documentation flags it as a security risk to enable only in trusted environments. It moves the exposure from the network to the filesystem rather than removing it. The general shape of this problem, where a model-adjacent feature becomes a data path nobody inventoried, is the subject of our audit of agent data exfiltration channels.

Bind-mounting a host cache into the container is the common pattern and the wrong one for an enclave. It makes the deployment depend on the destination host's HF_HOME layout, which is exactly the thing you cannot inspect ahead of a change window, and it leaves you with two artifacts to move and sign instead of one.

Build the snapshot into the image on a connected staging machine:

FROM vllm/vllm-openai:v0.26.0

ENV HF_HOME=/opt/hf \
    HF_HUB_DISABLE_SYMLINKS=1 \
    HF_HUB_DISABLE_TELEMETRY=1

# Populate the cache at build time. `hf`, not `huggingface-cli`.
# This tree is what catches steps 3 and 5 when the local path does not.
RUN --mount=type=secret,id=hf_token \
    HF_TOKEN=$(cat /run/secrets/hf_token) \
    hf download google/gemma-4-31B-it --revision <pinned-sha>

# Materialize the same revision as a plain directory to point --model at.
RUN --mount=type=secret,id=hf_token \
    HF_TOKEN=$(cat /run/secrets/hf_token) \
    hf download google/gemma-4-31B-it --revision <pinned-sha> \
      --local-dir /opt/models/gemma-4-31b-it

# Runtime is fully sealed.
ENV HF_HUB_OFFLINE=1 \
    HF_HUB_DISABLE_UPDATE_CHECK=1 \
    VLLM_NO_USAGE_STATS=1 \
    DO_NOT_TRACK=1
RUN mkdir -p /root/.config/vllm && touch /root/.config/vllm/do_not_track

Three things in that snippet are where published runbooks go stale.

huggingface-cli no longer exists. It was renamed to hf in huggingface_hub v0.34 (July 2025) and the legacy binary was fully removed in v1.0. Nearly every air-gap tutorial still ranking prints huggingface-cli download. If you copy one, your build fails on a connected machine, which is the good outcome, or you discover it inside the enclave, which is not.

HF_HUB_DISABLE_SYMLINKS=1 matters more than it looks. By default huggingface_hub creates symlinks in the cache and points snapshot directories at blobs. Set the variable and it duplicates the files instead. If your snapshot is staged on a NAS, written to removable media, or passes through a non-Linux box on the way in, symlinks do not survive the trip and you get a snapshot that looks complete and resolves to nothing.

You have to copy more than the hub tree. The defaults (official Hugging Face docs):

On performance: HF_HUB_ENABLE_HF_TRANSFER is deprecated now that the Hub runs on the Xet backend, and the current switch is HF_XET_HIGH_PERFORMANCE. HF_HUB_DISABLE_XET=1 forces the legacy path if your staging environment needs it.

One extra win while you are at it: ~/.cache/vllm/torch_compile_cache holds the FX graphs and Triton kernels vLLM produces, and the directory can be copied wholesale into the deployment to skip cold-start compilation. Warm it on staging hardware that matches the target. VLLM_DISABLE_COMPILE_CACHE=1 turns it off and VLLM_COMPILE_CACHE_SAVE_FORMAT=unpacked writes readable artifacts if you need to inspect what you are shipping.

HF_HUB_OFFLINE=1 does two specific things (official Hugging Face docs). It makes any HfApi call raise OfflineModeIsEnabled, and it makes file resolution read from the local cache only, raising an error when a file is missing. The important part is subtler: normally, even a fully cached file triggers an HTTP metadata check to confirm the revision. HF_HUB_OFFLINE is what skips that check. A cache that looks complete still generates traffic without it.

TRANSFORMERS_OFFLINE is a legacy alias, not a superset. huggingface_hub checks either variable and treats them identically, both forcing local_files_only=True. Several third-party guides imply a division of labor between the two, where one covers the hub and the other covers transformers. It does not work that way. Set HF_HUB_OFFLINE, and recognize TRANSFORMERS_OFFLINE when you meet it in an old runbook.

Now the part that matters most: neither variable is a network control. They govern huggingface_hub. They do not touch stats.vllm.ai, the Swagger CDN, the PyPI update check, NLTK corpus downloads, ModelScope, or per-request media fetches. Telemetry has its own switch, HF_HUB_DISABLE_TELEMETRY=1, which is separate from offline mode and applies across transformers, datasets, diffusers, and gradio.

There is one operational detail worth internalizing. HF_HUB_ETAG_TIMEOUT and HF_HUB_DOWNLOAD_TIMEOUT both default to 10 seconds. In an enclave whose firewall drops packets rather than rejecting them, a missed offline setting does not produce an error. It produces a 10 second stall per call, then continues. That is precisely how misconfigured offline setups reach production: they work, slowly, and nobody investigates a slow startup. Configure your enclave firewall to REJECT during testing so failures are loud, then switch to DROP for production if policy requires it.

There is a recurring class of bug where everything above is correct and vLLM tries to download anyway. The current reference case is vLLM issue #39039, filed 5 April 2026 against vLLM 0.19.0 with google/gemma-4-31B-it: vLLM attempts a Hugging Face cache-file download during inference despite a local model path, and setting HF_HUB_OFFLINE=1 makes it crash rather than proceed. The issue is open. The reporter did not identify which file is being fetched, and no maintainer resolution is present, so do not trust any summary that names one. The reporter does note the same setup worked for Llama, Mistral, and earlier Gemma generations.

That last detail is the whole lesson. This is not one bug. Equivalent reports go back through vLLM's tracker to 2023, resurfacing each time a new architecture lands: a model generation ships, its config or processor path resolves through a branch nobody exercised offline, and the offline flag converts a silent download into a hard crash. It regenerates rather than gets fixed.

So stop treating egress-denial as a state you configure once. Two rules follow:

Pin the vLLM version and the model together, as a pair, in version control. Not "vLLM 0.26.x" and "the latest Gemma", but one exact version and one exact revision SHA, changed together, tested together. vLLM's stable release is 0.26.0 (25 July 2026), and the cadence over the preceding two months (0.25.1 on 14 July, 0.25.0 on 11 July, 0.24.0 on 30 June, 0.23.0 on 13 June, 0.22.1 on 5 June) tells you how fast the surface underneath you moves.

Re-run the egress test on every model bump, not just every version bump. The failure is architecture-specific, so a new model on a known-good vLLM build is exactly as risky as a new vLLM build on a known-good model.

Getting model and version pinning right also happens to be where the sizing work lives, since a bump can change your memory budget as quietly as it changes your egress profile. Our walkthrough of reading vLLM's memory equation before touching a flag covers that side, and GPU sizing for large MXFP4 models under vLLM covers what happens when the pinned pair changes hardware requirements.

The inference server is rarely alone in there. Two patterns are worth knowing before you find them yourself.

Document parsing is where offline breakage hides. Open WebUI ships only the punkt_tab NLTK resource in its Docker image; averaged_perceptron_tagger_eng is not bundled. In a fresh container without network access, the first PPTX or DOCX upload fails on a missing NLTK resource and the job hangs at status=pending rather than surfacing an error. This is community-reproduced behavior, documented across several issues and a discussion in the project's tracker by multiple independent reporters, not vendor-documented behavior, and the proposed fix (baking both resources into the image with nltk.download('punkt_tab') and nltk.download('averaged_perceptron_tagger_eng')) is a proposal, not a shipped change.

The generalizable point is that the model is the part everyone remembers to stage. Tokenizer corpora, OCR language packs, spaCy models, layout-detection weights, and font files are the parts that are not, and they fail at first use rather than at boot. Audit the document path with a real file of every format you accept.

Ollama's offline story is simpler. The blob store under ~/.ollama/models is plain files: blobs plus manifests. Pull on a staging machine, copy the directory, done. What still needs work is the binary itself (stage the release rather than piping an install script), the manifests for every tag you intend to serve, and everything layered on top. Claims about Ollama telemetry environment variables circulate widely and contradict each other across sources, and we could not confirm the commonly cited variable against current documentation. Do not paste one into a hardened image on the strength of a blog post. Close egress at the network layer and verify. If you are still choosing between engines for the enclave, our Ollama vs vLLM comparison covers where each one earns its place.

Everything above is a claim until you test it. The test is cheap and belongs in CI, not in the change window.

Run the cheap variant first. --load-format dummy skips weight loading entirely, which means it resolves config, tokenizer, and processor paths without moving hundreds of gigabytes. A broken resolution path fails in seconds instead of after a full copy:

# Stage 1: does resolution work with no network and no weights?
docker run --rm --network none --gpus all \
  -e HF_HUB_OFFLINE=1 \
  -e VLLM_NO_USAGE_STATS=1 \
  -e HF_HUB_DISABLE_UPDATE_CHECK=1 \
  my-registry/vllm-sealed:0.26.0 \
  vllm serve /opt/models/gemma-4-31b-it \
    --served-model-name gemma \
    --tokenizer /opt/models/gemma-4-31b-it \
    --load-format dummy \
    --enable-offline-docs \
    --max-model-len 8192

If that starts and reports the model as ready, resolution is clean. Then run stage 2 with real weights and assert on a completion:

# Stage 2: full weights, no network, must serve a real token.
# No -p: with --network none there is nothing to publish. Assert from inside.
docker run -d --name vllm-airgap-test --network none \
  --gpus all \
  --env-file ./airgap.env \
  my-registry/vllm-sealed:0.26.0

docker exec vllm-airgap-test curl -sf \
  -X POST http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"gemma","prompt":"ping","max_tokens":4}' \
  | grep -q '"text"' && echo "PASS" || echo "FAIL"

--network none gives the container a namespace with nothing in it but loopback, so any egress attempt fails immediately rather than timing out. That is deliberate: you want the loud version of the failure in CI. Production will have DROP semantics and a 10 second stall per attempt, which is the version that hides.

Two additions make the gate genuinely useful. First, hit /docs in the test and assert the page renders content, because that is the only way you catch the missing --enable-offline-docs (default False) before a user reports a blank page. Second, if you serve multimodal, send a request containing a remote image URL and assert that it is refused rather than attempted, since that path is request-driven and no boot-time check reaches it.

Wire this as a required check on the branch that holds your pinned version pair. That is the difference between an egress policy and an egress hope. It is the same gate we build into on-premise inference deployments at Particula Tech, alongside the network policy, because in a regulated enclave the audit question is not "is it configured" but "when did you last prove it". Teams weighing that operational load against a managed alternative will find the tradeoff laid out in our cloud versus on-premise AI security and cost analysis, and the regulatory side in our notes on EU AI Act data sovereignty and residency.

The instinct after a successful air-gapped install is to freeze the image. Resist it. CVE-2026-22778 is a critical vLLM remote code execution flaw, CVSS 9.8, disclosed 2 February 2026, affecting versions 0.8.3 through 0.14.0 and fixed in 0.14.1. It was reachable through the multimodal endpoint, which is the same surface as the per-request media URL fetch discussed above. An enclave running a frozen 0.13 image is not safer than a patched one; it is a machine with a known critical vulnerability and no update path. (Treat the version range and the CVSS score as the facts here. The exploitation-scale figures circulating in trade coverage are not from the advisory.) We covered the hardening posture in detail in our vLLM inference server hardening guide.

What you need instead is a re-baselining cadence: a defined interval at which a new image is built on the connected side, signed, carried in, and gated on the egress test before it replaces the running one. The right interval depends on your change control process and your risk appetite, and anyone quoting you an industry-standard number for it is guessing. What is not negotiable is that the interval exists and that the egress test is the gate on every re-import.

The one-screen version, ordered by build stage:

# --- Build stage (connected staging host) ---
export HF_HOME=/opt/hf
export HF_HUB_DISABLE_SYMLINKS=1     # snapshot survives NAS / non-Linux transfer
export HF_HUB_DISABLE_TELEMETRY=1
hf download <repo> --revision <pinned-sha>    # NOT huggingface-cli
# copy: $HF_HOME/hub, $HF_HOME/xet, $HF_HOME/assets, ~/.cache/vllm

# --- Runtime stage (airgap.env, baked into the image) ---
HF_HUB_OFFLINE=1                 # cache-only; skips metadata check on cache hits
HF_HUB_DISABLE_UPDATE_CHECK=1    # stops the hf CLI hitting PyPI
HF_HUB_DISABLE_TELEMETRY=1
VLLM_NO_USAGE_STATS=1            # closes https://stats.vllm.ai
DO_NOT_TRACK=1                   # cross-vendor equivalent
# plus: touch ~/.config/vllm/do_not_track   (survives a bad env file)
# leave VLLM_USE_MODELSCOPE unset          (second registry, separate egress)

# --- Serve flags ---
# --tokenizer <same local path>   resolution cannot fall back to a repo id
# --enable-offline-docs           default False; /docs is blank without it
# --load-format dummy             stage-1 smoke test, no weight copy

# --- Gate (CI, every version AND every model bump) ---
# docker run --network none ... && assert a completion, assert /docs renders

The summary is short. HF_HUB_OFFLINE=1 is cache discipline, not a firewall. Egress-denial is a property you test on every bump, not a state you configure once. And the artifact that goes through the door should be one signed image that has already passed the test, not a weights directory and a hope. More on model selection and serving sits in our LLMs and models pillar guide.