Migrating AI Agents Between Models: 4 Silent Breaks

In July 2026 the team at Ploy published a same-agent comparison after moving a production agent from Claude Opus 4.8 to GPT-5.6 Sol, which OpenAI released publicly on July 9, 2026. Their published numbers:

Read that table as a case study, not a benchmark. It is one team, one agent, one workload, and it says nothing reliable about what your agent will do. The durable content of that writeup is not the deltas, it is the failure modes they hit on the way: 52% to 64% of the new model's file reads returned empty, and first-call cache hit rate started at roughly 0%.

Both of those are structural. Both will happen to you. Neither throws.

Four things, in rough order of how much damage they do before you notice:

The rest of this post takes them one at a time.

You cannot tell a regression from noise without a baseline, and you cannot build the baseline after you have already migrated. Do this first, on the model you are leaving:

This is the same instrumentation evals-driven development asks for. If you already run it, migration is mostly a matter of pointing it at a second model.

Here is a read_file tool as most teams write it:

{
  "name": "read_file",
  "description": "Read a file from the workspace.",
  "input_schema": {
    "type": "object",
    "properties": {
      "path":   { "type": "string" },
      "offset": { "type": "integer" },
      "limit":  { "type": "integer" }
    },
    "required": ["path"]
  }
}

The intent is obvious to a human: pass path, and pass offset and limit only when you want a slice. The schema does not say that. It says those fields exist and are not mandatory, and "not mandatory" is not the same instruction as "omit unless needed."

One model reads that and sends {"path": "src/app.ts"}. Another reads it and sends {"path": "src/app.ts", "offset": 0, "limit": 100} on every call, because filling a field is a perfectly valid reading of a schema that offers it. If your handler treats limit as authoritative, it returns the first 100 lines of everything, or worse, an empty slice when the invented values land somewhere unhelpful. The tool call succeeds. The trace shows a green tool result. The agent reasons over nothing.

This is exactly the class of bug that makes agents pick the wrong tool at scale: the model is not malfunctioning, the contract is underspecified.

{
  "type": "function",
  "name": "read_file",
  "strict": true,
  "parameters": {
    "type": "object",
    "properties": {
      "path":   { "type": "string" },
      "offset": { "type": ["integer", "null"] },
      "limit":  { "type": ["integer", "null"] }
    },
    "required": ["path", "offset", "limit"],
    "additionalProperties": false
  }
}

This is the one that makes a cheaper model produce a bigger bill, and the reason is that the two providers' caching models are structurally different rather than superficially different.

Read the minimum-prefix row twice. A 3,000-token shared prefix caches on OpenAI and silently does not cache on Opus 4.8. No error, no warning, just cache_creation_input_tokens: 0 and a bill that does not match your spreadsheet.

The migration failure is usually one of two shapes. Porting Anthropic to OpenAI, teams carry over the explicit breakpoints, assume that is the whole story, and never set prompt_cache_key, so requests scatter across machines and the shared prefix never lands on the same cache twice. That is the mechanism behind the roughly 0% first-call hit rate in the Ploy writeup:

response = client.responses.create(
    model="gpt-5.6",
    prompt_cache_key="tenant-acme-support-agent",
    instructions=STABLE_SYSTEM_PROMPT,
    input=user_message,
)

Going the other direction, teams rely on automatic caching that no longer exists and never place a single cache_control marker, so nothing caches at all.

Either way, verify rather than assume. Both providers report cache activity in the usage object, and if reads stay at zero across repeated requests with an identical prefix, something in your prefix is changing between calls. The usual culprits are a timestamp or request ID interpolated into the system prompt, non-deterministic JSON serialization, or a tool list assembled in a different order per request. We wrote up a real instance of this in why your Anthropic cache hit rate collapsed.

One structural point worth internalizing: on Anthropic, tools render before system, which renders before messages. Changing your tool list invalidates everything downstream of it. This is why "add a tool mid-conversation" is a much more expensive operation than it looks, on either provider.

If your agent uses a reasoning model, the reasoning is not a cosmetic artifact you can drop between turns. How it is carried differs by provider, and the mismatch produces intermittent failures rather than consistent ones, which is the worst debugging profile there is.

OpenAI. You can let the service hold state, or carry it yourself. If you carry it, set store: false and preserve every output item, because reasoning items come back with an encrypted_content field that the next call needs:

history = [{"role": "user", "content": "Find the likely bug in this repo."}]

first = client.responses.create(
    model="gpt-5.6",
    store=False,
    input=history,
    reasoning={"effort": "medium", "context": "current_turn"},
)

history.extend(item.model_dump() for item in first.output)
history.append({"role": "user", "content": "Now patch it."})

second = client.responses.create(
    model="gpt-5.6",
    store=False,
    input=history,
    reasoning={"effort": "medium", "context": "all_turns"},
)

The docs are explicit that to use all_turns with store: false you must "preserve every output item, append the next user message, and replay the complete history." Filtering the output array down to just the text, which is exactly what a tidy-looking history helper tends to do, breaks this. If instead you lean on server-side state via previous_response_id, an expired or invalid pointer returns a previous_response_not_found error and the recovery is to start fresh with the full input context.

Anthropic. Different model entirely. Thinking blocks must be passed back unchanged on the same model, signature intact. Editing or reconstructing them fails. And the cross-model behavior is the sharp edge for migrations: thinking blocks sent to a different model are dropped from the prompt rather than rendered. That drop happens before pricing, so it also lowers your reported input token count, which means a migration can look like it improved token efficiency when it actually just discarded context.

The practical rule for both: treat the reasoning payload as opaque state that round-trips verbatim, and never write a history-trimming helper that assumes an output item is only worth keeping if it has readable text in it.

Every mature system prompt is partly a list of workarounds for one specific model. Those workarounds do not become neutral on a new model. They become active instructions to a model that no longer has the behavior they were compensating for.

Three patterns worth auditing before you cut over:

Forced progress scaffolding. Lines like "after every three tool calls, summarize progress" existed because some models went silent for long stretches. Current Opus models narrate more by default, and Anthropic's own migration guidance is to remove that scaffolding rather than keep it. Left in, it produces a wall of redundant status updates.

Aggressive tool language. "CRITICAL: you MUST use this tool" was written to overcome an older model's reluctance. On a model that follows instructions literally, it overtriggers, and the fix is to dial the language back rather than add guardrails on top.

The inverse problem. Recent models can be conservative about reaching for tools, subagents, and file-based memory, because those need an explicit decide-to-use step. The documented lever is not a louder system prompt: it is prescriptive trigger conditions in each tool's own description field, stating when to call it and not only what it does. "Call this when the user asks about current prices or recent events" measurably outperforms a description that just says what the tool returns.

There is a fourth, subtler one. Removing sampling controls changes how you get variety. temperature, top_p, and top_k are rejected outright on Anthropic's current frontier models, so a pipeline that used temperature to get design or copy variance needs that variance elicited through prompting instead. The standard replacement is to have the model propose several distinct directions and pick one, rather than sampling the same prompt repeatedly.

Ploy's writeup notes the same category from the other direction: their new model converged toward generic gridded layouts without stronger steering. Style is a migration surface, not a cosmetic afterthought.

Once the four breaks are addressed, the mechanical part is short:

One caution on step 2. Cheaper per token is not cheaper. Cache hit rate dominates the bill once you have a large static prefix, cache writes carry a premium, and a migration that breaks caching can erase the entire per-token saving while every dashboard still shows the lower rate.

Vendor migration guides document what now returns a 400. That is the easy half, because an error tells you where to look. The expensive half is the four failure classes that return HTTP 200 with a worse answer: optional parameters filled with invented values, caching that quietly stops working, reasoning state discarded by a helper that looked correct, and prompts still fighting a model that is no longer there.

Fix the schemas to be required-but-nullable, port the caching strategy deliberately instead of assuming it carries over, round-trip reasoning payloads verbatim, and audit the system prompt for workarounds that outlived their model. Then instrument, shadow-run, and keep the flag.

Deciding which model to move to in the first place is a separate question, and one worth answering with your own workload rather than a leaderboard. Our Claude Fable 5 versus Opus 4.8 decision guide works through that tradeoff, and the AI agents pillar covers the surrounding architecture. If you would rather have the regressions found before your users find them, model-evaluation and migration work is something Particula Tech runs against real agent traffic.