OpenAI Agents SDK With Ollama: What Still Calls OpenAI

First, a correction to a claim you will meet in older comparisons, including the original version of our own LangGraph, CrewAI and OpenAI Agents SDK comparison: the SDK is not limited to OpenAI models. Its models documentation names three built-in integration points for any OpenAI-compatible endpoint:

LiteLLM and Any-LLM adapters exist too (openai-agents[litellm], openai-agents[any-llm]), but the docs describe both as best-effort beta integrations. For Ollama or vLLM, which already speak the OpenAI wire format, the plain OpenAI client path has fewer moving parts.

Use Agent.model with OpenAIChatCompletionsModel, and disable tracing before anything else runs:

import asyncio

from openai import AsyncOpenAI
from agents import Agent, OpenAIChatCompletionsModel, Runner, set_tracing_disabled

# First line of the process, before any agent runs: no spans are created or exported.
set_tracing_disabled(True)

client = AsyncOpenAI(
    base_url="http://ollama.internal:11434/v1",  # vLLM: http://vllm.internal:8000/v1
    api_key="ollama",  # Ollama requires a value but ignores it
)
model = OpenAIChatCompletionsModel(model="llama3.2", openai_client=client)

agent = Agent(
    name="Policy reviewer",
    instructions="Summarize the clauses that changed and cite each one.",
    model=model,
)

async def main() -> None:
    result = await Runner.run(agent, "Compare the attached policy versions.")
    print(result.final_output)

asyncio.run(main())

Two choices in that block are deliberate. The model object is attached per agent rather than through set_default_openai_client, because that function (and set_default_openai_key) takes use_for_tracing=True by default, which hands the client's key straight to the trace exporter. The repository's own global-client example avoids this by passing use_for_tracing=False and also calling set_tracing_disabled(disabled=True). And Chat Completions is chosen explicitly; the section on Responses below explains why.

Ollama requires an api_key value and ignores it. One more egress note from Ollama's docs: through a signed-in local server you can select a cloud model with a :cloud suffix, such as gemma4:cloud. That request is served by Ollama's cloud, not your GPU, so model names belong in the same review as base URLs. If you are still choosing a server, our Ollama vs vLLM comparison covers that decision; the SDK wiring is identical for both.

The default tracing chain is one BatchTraceProcessor feeding one BackendSpanExporter. Both are created lazily on first use, so importing the SDK opens no network client. The exporter class carries the destination as a constant at line 46 of src/agents/tracing/processors.py:

_OPENAI_TRACING_INGEST_ENDPOINT = "https://api.openai.com/v1/traces/ingest"

The constructor accepts an endpoint argument, but the default exporter is built with no arguments, and nothing in the tracing package reads OPENAI_BASE_URL. The only other api.openai.com URL hardcoded in the package source at the tag is the realtime client's WebSocket URL, which is outside this post.

For each batch, the exporter groups items by their own tracing_api_key, then resolves a key in this order: the span's key, the exporter's stored key, then os.environ.get("OPENAI_API_KEY"). Only a found key is cached. The source comment says why: a key that appears later, "such as from a later load_dotenv()", is still picked up. A skipped batch at startup does not mean the next batch skips.

Row 2 is the easy one to misread. The two-variable setup, OPENAI_BASE_URL pointed at Ollama and OPENAI_API_KEY=ollama, lands there. The exporter does not validate the key before sending; it posts the batch and inspects the status afterwards. A 4xx other than 408, 409 or 429 is logged as a non-fatal client error and dropped without retry. The #5291 reporter described the same thing from the outside: a separate connection still goes to api.openai.com:443 and a tracing 401 is logged. A 401 is proof the request left, not proof that nothing did.

Row 3 is the one that stays invisible. A developer's shell, a shared .env.local that also serves a cloud prototype, or a secrets-store injection that gives every service the same OpenAI key: any of them flips a local-model agent into exporting prompts and tool outputs, and no log line marks the change.

Row 4 is documented behaviour. The SDK's tracing docs have a section on tracing with non-OpenAI models that passes os.environ["OPENAI_API_KEY"] to set_tracing_export_api_key() precisely so local-model runs show up in the OpenAI dashboard. Copying that snippet from the docs into a private deployment does exactly what it says.

There are six levers at v0.23.1. Two of them do not do what their names suggest.

The per-run flag deserves the detail. At v0.23.1, create_trace_for_run() returns early whenever a trace is already current, so a run started inside your own trace() block keeps producing task, agent, turn, guardrail, handoff and function spans, and function spans carry tool output. The issue (#5289) was closed on 2026-10-04 with a fix merged to main the same day (PR #5296), but that fix is not in v0.23.1. Use the process-level switch or a processor replacement, not the per-run flag.

add_trace_processor is the other trap. Its docstring says it adds a processor that receives all traces and spans; the docs state plainly that it leaves the default OpenAI exporter in place. Registering a local exporter that way gives you two destinations, not one. The docs also note that processors are independent observers, so a redaction processor registered ahead of an exporter does not stop that exporter from receiving data if redaction fails. The docs' answer is to keep redaction inside the exporter that writes.

import json

from agents import set_trace_processors
from agents.tracing.processor_interface import TracingExporter
from agents.tracing.processors import BatchTraceProcessor


class JsonlExporter(TracingExporter):
    """Append traces and spans to a local file that your on-prem log pipeline ships."""

    def __init__(self, path: str) -> None:
        self.path = path

    def export(self, items) -> None:
        with open(self.path, "a", encoding="utf-8") as f:
            for item in items:
                data = item.export()
                if data:
                    f.write(json.dumps(data, default=str) + "\n")


# Replaces the default OpenAI exporter. Call before any agent runs or trace() opens.
set_trace_processors([BatchTraceProcessor(JsonlExporter("/var/log/agents/traces.jsonl"))])

# Agent host environment (systemd unit, Kubernetes env, deployment manifest)
export OPENAI_AGENTS_DISABLE_TRACING=1
export OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA=false
# The exporter falls back to this variable for every batch; keep it off hosts that only call local models
unset OPENAI_API_KEY

The reason to run the SDK against Ollama or vLLM in banking, insurance, healthcare or pharma is that prompts, source documents and tool outputs may not leave the perimeter. The default exporter breaks that condition silently as soon as any OpenAI key is resolvable on the host. For anyone with a data-residency requirement or an outsourcing register, that is a data-egress finding with a third-party recipient, not a logging preference.

The SDK docs add one relevant line: "Tracing is unavailable for organizations that use OpenAI's APIs under a Zero Data Retention (ZDR) policy." It is a documented limitation of OpenAI's side, not a control you can set or verify from inside your network. The controls that you can verify are below.

The controls we recommend for agent hosts, in order of how much each one proves:

The same audit applies one layer down and one layer across. Serving stacks have their own phone-home paths, covered in blocking Hugging Face calls in an air-gapped vLLM deployment, and coding agents have theirs, covered in OpenCode's offline mode with local models. Local weights do not mean local traffic in any of them until someone checks.

The SDK calls the Responses API by default. Its models docs say plainly that many providers do not support it and that you may see 404s, and give two fixes: set_default_openai_api("chat_completions"), which works when you configure OPENAI_API_KEY and OPENAI_BASE_URL through environment variables, or OpenAIChatCompletionsModel as in the wiring block above.

The local servers are further along than that warning suggests. Ollama's compatibility docs at v0.40.4 list /v1/responses as added in v0.13.3, with tools, streaming, instructions and reasoning supported, but only the non-stateful flavor: no previous_response_id and no conversation support. vLLM v0.31.0 registers POST /v1/responses along with retrieval and cancel routes. So a 404 on Ollama today points to an older server or a proxy in front of it rather than a missing feature.

Use Chat Completions with the SDK anyway. It is the path the SDK docs use for non-OpenAI providers, and Ollama's Responses endpoint is stateless, so the server-side conversation features that make Responses attractive are not there to use.

The strict validation flag is worth turning on in regulated builds. The docs say routing through Chat Completions silently drops Responses-only fields, including previous_response_id, conversation_id, the Responses prompt field and non-text tool outputs. Pass the provider as RunConfig(model_provider=provider), or set openai_strict_feature_validation=True on a MultiProvider. A silent drop in an agent that a reviewer later signs off is worse than a loud error at startup.

One more caveat for Ollama: its Chat Completions compatibility does not support tool_choice. An agent that depends on forcing a specific tool needs a different design on that server. If tool calls come back empty from vLLM instead, check server-side parsing first, covered in empty tool calls and the vLLM reasoning parser.

When an agent has an output_type, the Chat Completions path converts it to a response_format of type json_schema, named final_output, with strict taken from the schema. It sends that on every model call in the run, not only the last one. AgentOutputSchema(YourType, strict_json_schema=True) is the default; wrapping your type with strict_json_schema=False is the escape hatch the SDK's own error message suggests for schemas strict mode cannot express.

The SDK docs give an example of a provider that accepts JSON output but not a schema, returning BadRequestError: Error code: 400 with 'response_format.type' : value is not one of the allowed values ['text','json_object']. That is not what current Ollama does. At v0.40.4 Ollama maps a json_schema response format to its own format constraint on both the Chat Completions and Responses endpoints.

The problem that bites on vLLM is different. Issue #3709, filed against SDK 0.17.7 with vLLM serving a Gemma model, reported that sending output_type on every turn suppresses tool calls, because the server constrains decoding to the schema from the first token. A maintainer closed it on 2026-06-30, confirming the SDK passes the schema on each invocation and calling the conflict a vLLM limitation. The related pull request #3710, which proposed deferring structured output until the final turn, was closed without merging.

The recommended pattern is two stages:

That split costs one extra model call per run. It also produces a cleaner audit trail: the evidence-gathering step and the structured conclusion are separate spans that a reviewer can check independently. For server-side guided decoding failures on vLLM itself, see vLLM structured output not working.

Run grep -r OPENAI_API_KEY across your agent hosts' unit files, pod specs and dotenv files, and remove the key anywhere the agents only call Ollama or vLLM. Add set_tracing_disabled(True) or a set_trace_processors call as the first statement of each entry point. Then add a deny rule for api.openai.com on the agent subnet and alert on any [non-fatal] Tracing line in agent logs. If that alert ever fires, an exporter tried to send spans, and you know which host to look at. More patterns for running agents in private environments sit in our AI agents pillar.