Long-Running Agents: Fix Context Bloat With Compaction

The Failure Mode: Why Long-Running Agents Overflow or Degrade

There are two distinct failure modes, and conflating them leads to the wrong fix.

The first is the hard crash. Total tokens exceed the model's window and the API returns context_length_exceeded. This is binary and obvious. It tends to hit agents that do a lot of file reads, web fetches, or large API calls, because each of those drops a big payload into history. A 200k-token window sounds enormous until a single document read is 40k tokens and you do four of them.

The second is the soft degradation, which is sneakier and more common. The agent does not crash. It just gets worse. Long before you hit the limit, performance starts to slide because the model has to find the relevant needle in a haystack of stale tool output. This is the "context rot" effect: as the window fills, retrieval and reasoning over that context degrade even though you are technically within limits. We covered the evidence in what Chroma's context rot study proves about long context windows, and the practical takeaway is that a full window is a degraded window. You do not get to use all 200k tokens at full quality.

Measuring Token-Per-Turn Growth

You cannot fix bloat you cannot see. Before adding any compaction logic, instrument the loop to log, on every turn, the running total of prompt_tokens and the per-turn delta. Most observability tools (Helicone, Langfuse, Traceloop) capture this automatically; if you are rolling your own, the token counts are in every API response.

What you are looking for is the shape of the curve. Healthy agents grow roughly linearly and slowly. Bloated agents show sharp step increases that line up with tool calls. When you see a single turn add 30k tokens, that is your culprit, and it tells you whether to summarize (the result was useful but verbose) or to offload (the result was a large blob the agent barely touched).

Here is a representative profile of a coding-style agent running without compaction, the kind of growth that ends in a crash:

Notice that the model's own reasoning is a rounding error here. The window is filled by tool results, and by turn 35 you are already past the 70% threshold where compaction should have fired. The fix is not a bigger window. It is a loop that stops carrying turn 5's file read all the way to turn 60.

The Compaction Trigger: Summarize at 70-80% Capacity

The core technique that all the major runtimes converge on is the same: trigger LLM summarization when the context reaches 70-80% of the window. Not 95%, not 100%. The headroom matters for two reasons. First, the summarization call itself needs room to run, you have to send the history to the model to summarize it. Second, the very next tool result might be large enough to overflow before compaction completes, so you need a buffer.

The structure of a good compaction is two zones:

Keep the active window verbatim. The most recent 2-4 turns stay at full fidelity. The agent is mid-thought on its current sub-task, and it needs high-resolution access to exactly what just happened: the last tool call, the last result, the last error. Compressing the active window is how you get an agent that forgets what it was doing two steps ago.

Replace the archive with a structured summary. Everything older than the active window gets condensed into a single summary message. This is not a vague paragraph. It is a structured record: what the overall goal is, what has been accomplished, what files or resources are in play, what errors are still open, and what the next step was going to be. The discipline is that the summary captures decisions and state, not the verbose record of how each step was executed.

This is a specialized application of the broader prompt compression techniques for context windows, but agents add a constraint that static prompts do not have: the compaction has to be reversible enough that the agent can keep acting. A summary that reads well but drops the open file path is worse than no summary at all.

The Three Converged Techniques: Anthropic, Claude Code, LangGraph

By mid-2026 the leading agent stacks have settled on overlapping patterns. Knowing all three lets you pick the right one for your runtime instead of reinventing it.

The pattern to internalize: offloading is lossless and should be your first move for large payloads; summarization is lossy and should be reserved for the historical narrative of completed steps.

Offloading Large Tool Results to the Filesystem

Summarization throws information away. For a 40k-token document the agent might need to re-read later, that is the wrong trade. The better move is to offload: write the full payload to disk (or an object store), and put a short reference in context instead.

The rule of thumb that has converged across stacks: if a tool result exceeds roughly 20,000 tokens, do not inline it. Instead:

This flips the economics. Instead of paying for 40k tokens on every subsequent turn, you pay for a 50-token reference, and only re-load the full thing in the rare case the agent actually needs it again. For documents, search corpora, large API responses, and file reads, offloading is strictly better than summarization because nothing is lost.

There is a design subtlety. The reference has to carry enough metadata for the agent to decide whether to re-read. "Tool result saved to disk" is useless. "Search returned 10 results about authentication middleware, saved to turn_34_search.json, top result: rate-limiter.ts" lets the agent reason about whether it needs the full payload without spending the tokens to load it. Treat the inline reference as an index entry, not a placeholder.

This is also where filesystem offload meets retrieval: the workspace becomes a small, agent-owned knowledge store. If your agent's offloaded results start looking like a real corpus, the context engineering practices for managing what the agent sees become relevant, because re-reading by path is the simplest possible form of agent-controlled retrieval.

Checkpointing State to Git and Progress Files

Compaction is lossy by design, which means you need a source of truth that lives outside the lossy summary. That source of truth is a durable checkpoint: a git commit, a progress file, or both.

For coding agents, git is the natural checkpoint. After each meaningful step, commit. The diff and commit history become a perfect, full-fidelity record of what changed, completely independent of the conversation. When you compact the context, you can safely drop the verbose record of file edits because git log and git diff reconstruct the actual state. The summary says "implemented the auth middleware, committed as a1b2c3d"; the agent can always git show a1b2c3d to see exactly what it did.

For non-coding agents, a progress file plays the same role. Maintain a structured file, progress.md or state.json, that the agent updates as it works: tasks completed, decisions made, current objective, blockers. This file is never summarized, it is the ground truth. When compaction runs, the summary can point at the progress file rather than trying to encode everything itself.

The combination is powerful: compaction keeps the working context small, and the checkpoint guarantees you can rebuild full state after a crash or a bad summary. If the agent dies at step 90, it does not start over, it reads the progress file and the git history and resumes. This durability mindset connects directly to how agents persist and recall context across sessions; compaction handles the live context, while checkpoints handle survivability.

Avoiding Summary-Induced Quality Loss

This is the section that separates agents that survive compaction from agents that quietly fall apart. Summarization is lossy. The entire game is making sure it loses only the safe-to-lose information.

Never compress the active working set. Anything the agent will reference or act on in the next few steps must survive verbatim. That includes open file paths, resource IDs, the exact current task definition, unresolved errors, and any structured data (a list of records, a config object) it will use again. The summary is for completed history, not for the live task.

Preserve specifics over narrative. A bad summary says "the agent explored the codebase and made progress on the authentication feature." A good summary says "goal: add JWT auth to /api/login. Done: created auth/jwt.ts, added middleware to routes.ts (commit a1b2c3d). Open: token refresh not implemented, test in auth.test.ts line 42 failing on expired-token case. Next: implement refresh in jwt.ts." The second one lets the agent resume; the first one forces it to re-discover everything.

Keep what you cannot reconstruct, drop what you can. This is the deciding heuristic. The full text of a file you wrote? Droppable, git has it. A test failure message you have not addressed? Keep it, it is not recorded anywhere else. The reasoning behind a key architectural decision? Keep it, the agent will not re-derive it. Apply this test to every category of information before deciding whether it survives compaction.

The reason this matters so much circles back to the opening statistic: about 65% of enterprise agent failures come from context drift and memory loss. Compaction done carelessly is a direct cause of exactly that failure. Compaction done with these guardrails is the cure. The difference is entirely in what you choose to preserve.

Putting It Together: A Compaction Loop

Here is the full pattern assembled, the loop logic a production long-running agent should run:

Start with measurement and offloading, those two alone eliminate most overflow crashes with zero accuracy risk because offloading is lossless. Add summarization only once you have a sense of which information is safe to compress, and add checkpoints before you rely on compaction in anything that runs for hours.

At Particula Tech, when we audit a long-running agent that overflows or degrades mid-task, this is the order we work in: instrument the token curve first, then offload the big payloads, then add the compaction trigger, then prove durability with checkpoints. The crashes usually stop after step two; the quality recovers after step six. A bigger context window is almost never the answer, because a full window is a degraded window, and the bloat just returns at a higher ceiling. The fix is a loop that knows how to forget the right things.