How to Secure an Ollama Server Exposed to the Internet

OLLAMA_HOST does double duty, and getting that straight prevents most accidental exposure. On the server it is the listen address. In the CLI and the client libraries it is the URL to connect to. The same variable exported in a shell profile so that ollama list can reach a remote box will, on a machine that also runs ollama serve, change what the server binds. Set it deliberately in exactly one place on a serving host, the systemd unit override, and nowhere else.

# sudo systemctl edit ollama.service
# This is the shape of nearly every exposed host in the scan:
#   Environment="OLLAMA_HOST=0.0.0.0:11434"
#
# This is what a box behind a proxy should carry instead:
[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_NO_CLOUD=1"
Environment="OLLAMA_MAX_QUEUE=64"
Environment="OLLAMA_NUM_PARALLEL=2"

OLLAMA_NO_CLOUD=1 belongs there from day one on a private deployment. Ollama can route a request to a hosted model when the model name resolves to one, and on a box that exists specifically so prompts stay inside the perimeter, that is the one silent failure mode worth closing before anybody types a model name. With the variable set, a cloud model reference fails instead of leaving the building.

Then verify the bind by observation rather than by reading the unit file back:

ss -ltnp | grep 11434
# Good:  LISTEN 0 4096 127.0.0.1:11434 0.0.0.0:*  users:(("ollama",pid=...))
# Bad:   LISTEN 0 4096   0.0.0.0:11434 0.0.0.0:*
# Also bad, and easy to miss: *:11434 on a dual-stack listener.

The throughput and operations tradeoff behind choosing that server is in our Ollama and vLLM comparison; this post assumes the box already exists.

The scale is the argument. A single measured pass over the public internet, published in January 2026, produced this:

Two distributions inside that data matter more than the headline. First, activity is extremely concentrated: 13 percent of hosts produced 76 percent of all observed activity, while the 36 percent of hosts seen exactly once contributed under 1 percent. Most exposed endpoints are transient lab boxes, and a small stable minority is doing real work in public.

Second, 56 percent sat on fixed-access telecom networks rather than cloud hosting. Read that as workstations, home labs and office desktops rather than fleets. If your exposure hunt only covers the server inventory, it will cover under half of the realistic surface. Every laptop that ran a quickstart and every GPU box under somebody's desk belongs in the same scan.

The resale economics close the loop. The same research describes an operation that scans for exposed Ollama, vLLM and other OpenAI-compatible endpoints, validates each one by the quality of the responses it returns, and sells access to the working ones. That validation step is why an exposed endpoint does not stay unnoticed: the scanning is continuous, the qualification is automatic, and a host that answers well is inventory.

Be precise about the mechanism, because the loose version of this claim is wrong. When a caller passes tool definitions to /api/chat, the model returns a structured tool call and the caller executes it. The exposed Ollama process does not run your shell command. So why does the 48 percent figure change the severity rating?

Three reasons, and none of them requires the server to execute anything.

An endpoint that can drive an agent loop is worth more. Tool calling is the capability that turns a stolen text generator into a stolen automation backend, which is exactly why the capability gets recorded and priced during validation. Your electricity and your GPU hours are the cheap part of what is being taken.

The write routes are unauthenticated on the same port. This is the real server-side exposure and it has nothing to do with tools. /api/create installs a model with an attacker-written TEMPLATE and SYSTEM block. That text is prepended to every request served by that model name, which means the 201 hosts running guardrail-stripped templates are a demonstration of a general capability: anyone who can reach 11434 can change what your model is instructed to do, permanently, without touching your application code. If your own agents call that host, the instructions land in your loop.

Two routes make outbound connections. /api/pull and /api/push are the only documented endpoints that reach an external network, and both are open. Pull makes your host fetch content of the caller's choosing until the disk is full. Push, on a host with registry credentials configured, moves data out.

The OpenAI-compatible routes are the trap in that table. A proxy rule written as a regex over /api/ looks complete and forwards the entire /v1/ tree untouched. Write the allowlist as an allowlist.

There is exactly one check worth trusting, and it is a connection attempt from somewhere else.

# From a machine on a different network. Not the host. Not the same LAN.
nmap -Pn -p 11434 ollama.example.internal

# Then read the body, because a closed-looking scan and a talkative API disagree often.
curl -s -m 5 http://ollama.example.internal:11434/api/tags     # your model inventory
curl -s -m 5 http://ollama.example.internal:11434/api/ps       # what is loaded right now
curl -s -m 5 http://ollama.example.internal:11434/api/version  # server version

Now the five controls that get mistaken for that check, and what each one actually covers:

The Docker row is the one that survives code review, because the firewall config reads correctly. Docker documents it plainly: traffic to and from a container with published ports is routed in the nat table and diverted before it reaches the chains ufw filters on. docker run -p 11434:11434 therefore publishes on every interface no matter what your deny rule says. We covered the same mechanism in detail in self-hosted vector database security hardening, and it applies identically here. Publish to loopback with -p 127.0.0.1:11434:11434, or use an internal container network with no host mapping and let the proxy container be the only thing that can reach it.

# What is actually published, from the host:
docker inspect -f '{{json .NetworkSettings.Ports}}' ollama
# {"11434/tcp":[{"HostIp":"0.0.0.0","HostPort":"11434"}]}   <- exposed
# {"11434/tcp":[{"HostIp":"127.0.0.1","HostPort":"11434"}]} <- fine
# null                                                       <- better

Ollama has no concept of a caller, so the proxy has to supply one. The minimum useful configuration terminates TLS, authenticates every request, rate limits per identity rather than per IP, denies the write routes outright, and logs enough to answer an audit question later.

limit_req_zone $http_authorization zone=ollama:10m rate=30r/m;

server {
    listen 443 ssl;
    http2 on;
    server_name ai.internal.example;

    ssl_certificate     /etc/ssl/certs/ai.internal.example.crt;
    ssl_certificate_key /etc/ssl/private/ai.internal.example.key;
    ssl_protocols       TLSv1.3;

    # Write and registry routes are never reachable through the proxy.
    location ~ ^/api/(create|copy|delete|pull|push|blobs) { return 403; }

    location / {
        auth_request     /_authz;
        limit_req        zone=ollama burst=10 nodelay;
        client_max_body_size 1m;
        proxy_pass       http://127.0.0.1:11434;
        proxy_read_timeout 300s;
        proxy_buffering  off;   # keep token streaming intact
    }

    location = /_authz {
        internal;
        proxy_pass              http://127.0.0.1:8081/verify;
        proxy_pass_request_body off;
        proxy_set_header        Content-Length "";
    }

    log_format ollama escape=json
        '{"ts":"$time_iso8601","sub":"$upstream_http_x_subject","path":"$uri",'
        '"status":$status,"bytes":$body_bytes_sent,"ms":$request_time}';
    access_log /var/log/nginx/ollama.jsonl ollama;
}

Two details in there are load-bearing. proxy_buffering off keeps streaming responses streaming, and teams that omit it usually conclude the proxy broke Ollama and remove the proxy. client_max_body_size 1m is the cheapest defence against oversized multimodal payloads, tuned up deliberately if you serve images.

And the obvious one that gets skipped: this configuration is decorative while 11434 is still published on a routable interface. The proxy is not a filter placed in the path, it is the only path, and that is true only if the upstream is on loopback or on an internal network.

An inference host has almost no legitimate reason to originate outbound connections. Denying them by default converts several classes of future problem, model-supply-chain fetches and any request-forgery path included, into a connection timeout.

# /etc/nftables.conf, native (non-container) Ollama install
table inet filter {
    chain output {
        type filter hook output priority 0; policy drop;
        ct state established,related accept
        oif "lo" accept
        ip daddr 10.0.0.0/8 udp dport 53 accept    # internal DNS
        ip daddr 10.0.0.0/8 udp dport 123 accept   # internal NTP
        ip daddr 10.0.0.0/8 tcp dport 9100 accept  # metrics scrape target
    }
}

What breaks is short and predictable: ollama pull and ollama push, which are the only documented routes that reach an external network. Inference on models already in the store is unaffected. Cloud model routing is already off if you set OLLAMA_NO_CLOUD=1.

The operational replacement is a promotion step, treated the same way you treat container images. Pull the model on a staging host with egress, record its manifest, then move the blob directory to the serving host or serve it from an internal mirror. Model updates become change-controlled artifacts with an approver.

One trap repeats the ingress lesson exactly. If Ollama runs in a container, its outbound traffic is filtered in the forward path rather than in the host OUTPUT chain, so an OUTPUT policy of drop looks decisive and leaves container egress completely untouched. Filter container egress in the forward path, or give the container a network with no default route and reach the registry through an explicit proxy.

ollama pull qwen3:8b resolves a tag, and a tag is mutable in the same way :latest is mutable. Underneath, the layers are content-addressed sha256 blobs with one manifest per tag, which gives you something stable to record even though the human-readable reference is not.

The part most teams have never inspected is the template. A model's TEMPLATE, SYSTEM and PARAMETER lines determine how every message is framed before the model ever sees it, which makes them policy artifacts, not packaging details. They are also exactly what an /api/create call rewrites, and the 201 hosts observed serving templates with safety instructions removed are what that looks like when nobody is checking.

# At promotion time, on the staging host:
ollama show --modelfile qwen3:8b > /etc/ollama/baseline/qwen3-8b.modelfile

# Record the manifest too. Adjust the path for your OLLAMA_MODELS setting;
# a systemd install keeps it under the ollama service user's home.
sudo sha256sum /usr/share/ollama/.ollama/models/manifests/registry.ollama.ai/library/qwen3/8b

# On every serving host, as a deployment gate that fails the rollout:
diff <(ollama show --modelfile qwen3:8b) /etc/ollama/baseline/qwen3-8b.modelfile || exit 1

Three rules make that stick. Pull only through the change-controlled path, never on a serving host. Deny /api/pull and /api/create at the proxy so the only way a model changes is the approved one. And run the diff on a schedule as well as at deploy, because the value of the check is detecting a change nobody deployed.

Ten checks. Not one of them is satisfied by reading a config file.

If you do one thing this week, do the second item. Take every host in the estate that has ever had Ollama installed, laptops and lab boxes included, and run nmap -Pn -p 11434 against it from a machine on a different network. Curl /api/tags on anything that answers. For most estates that is a two-hour job, and it is the only step here that tells you whether the rest of the list is urgent. For the wider control set around private model serving, start at our AI security pillar.