AI Agent Tool Failure Recovery: Retry, Re-route or Replan

A June 2026 benchmark called ToolBench-X names the ways a tool can betray its caller. Starting from clean tool environments, it injects five structured hazard types: Specification Drift, Invocation Error, Execution Failure, Output Drift, and Cross-source Conflict. One design constraint bounds everything it can tell you: each injected instance remains solvable through at least one valid recovery path, such as retrying, fallback, verification, or cross-checking.

The final benchmark contains 1,106 instances, of which 378 are sequential, 358 parallel and 370 mixture tasks. A task counts as correct if the execution state recorded by the backend at task completion matches the ground truth, or if the model's final response explicitly contains the ground-truth answer. Twelve models appear in the main results table.

Two of the five arrive on the channel your code already reads. Two never do. The fifth, specification drift, lands on either side depending on whether the change was structural or semantic, and gets a section of its own later.

Now the part that contradicts the obvious guess. Across all twelve models the ordering of the hazard columns is stable, and the column agents score highest on is Output Drift, spanning 0.330 to 0.750. The column they score lowest on is Invocation Error, spanning 0.127 to 0.353. Cross-source Conflict is the second hardest family, spanning 0.137 to 0.460.

Overall accuracy for those two models is 0.453 and 0.212, and the two columns rank the five hazards identically.

The hardest column is the one where the tool rejects the agent's own call, which is the case the model's own built-in correction already targets. The tool-use documentation says that when a tool request is invalid or missing parameters, the model retries two to three times with corrections before apologising to the user. That ladder fires, and invocation error is still the worst column in the table. A louder error does not make a hazard easier.

Do not over-read that result in the other direction. What a benchmark injects is drift it can label, which means drift it can also make noticeable. Whether agents handle semantic corruption well, or whether injected drift is easier to spot than what a delayed nightly batch produces, this paper does not settle.

ToolMaze, published in June 2026, comes at the same problem from the planning side. To separate systematic replanning from blind trial-and-error, it adopts a two-dimensional design: DAG-based topological complexity and a 2 x 2 taxonomy of tool perturbations, explicit or implicit crossed with transient or permanent. Its metric is Perturbation Recovery Rate, the conditional probability of resolving an encountered perturbation, whose denominator is the tasks where a perturbation occurred rather than all tasks.

Two findings matter. Driven by systemic over-trust in corrupted outputs, Perturbation Recovery Rate plummets by around 37 percent under implicit semantic failures, while complex topologies trap agents in futile trial-and-error loops. Read that 37 percent correctly: it is a relative drop between conditions, not a recovery level and not a share of tasks. The second belongs in a planning meeting. Agentic fault tolerance improves with model scale 3.66 times slower than basic task execution, restated in the body as each order-of-magnitude increase in model size being associated with roughly 3.66 times more gain in baseline task completion than in fault tolerance.

That is not a claim that models get worse at recovery. It is a claim about slopes: the gap between what a model does with cooperative tools and what it does with hostile ones widens as you scale, because the two curves climb at different rates. Waiting for the next model is a strategy for task completion and not one for tool failure recovery.

The futile loop at the end of that road has its own signature and its own guard, in stopping an agent that keeps calling the same tool without progressing.

ToolBench-X ran the comparison that settles what to spend on. On a 200-task subset across five models it measured four conditions: Baseline is the exception-injected setting, Oracle is the clean upper bound, and Test-time scaling and Hint are recovery strategies. The Oracle-to-Baseline gap on that subset runs roughly 35 to 50 percentage points, the price of the injected hazards.

Test-time scaling, meaning more inference budget, improves Baseline accuracy by only 3.5 to 11.5 percentage points. Hint, meaning telling the agent which hazard it faces, lifts Baseline accuracy by 25.5 to 35.5 absolute points, recovers roughly 60 to 80 percent of the lost accuracy across all five models, and beats test-time scaling by 24 to 32 points. The paper's own conclusion is the thesis of this post: these findings identify diagnosis, rather than recovery, as the primary bottleneck, and once an agent knows which hazard it faces it can often recover effectively.

So the move is not a longer ladder. Exponential backoff and the basic tool error handling recipe are solved and written up in making agents use tools correctly; adding rungs buys nothing against a hazard the ladder cannot see. The move is a classification step that runs before the response is chosen and writes the hazard name into the result the model reads.

One caution on the hint condition. In the benchmark it comes from a harness that knows what it injected. In your system nothing knows, which makes 25.5 to 35.5 points an upper bound on what a perfect label is worth and puts the real work in producing the label at all.

ToolMaze's 2 x 2 is more than a benchmark design. It is the smallest classifier that separates the responses you can code. Explicit against implicit asks whether the failure announced itself. Transient against permanent asks whether a second attempt on the same tool could plausibly return something different. Answer both and the response falls out.

Notice what is missing from the four recovery paths ToolBench-X names: retrying, fallback, verification and cross-checking. Stopping is not among them, because the benchmark guarantees by construction that recovery exists. Your tool inventory carries no such guarantee. That is why stop appears in the table as a response at all, and why it has to be reachable from code rather than from a prompt. Designing the escalation it hands off to is a separate post.

Two definitions keep the table honest. Re-route means calling a different tool that answers the same question from a different underlying record, which is why the implicit permanent row warns against a second source reading the same table through another view. Replan means rewriting the part of the dependency graph that has not executed yet, not re-running the node that failed; building and scheduling that graph is covered elsewhere.

Put the classification in the tool wrapper, not the system prompt. A prompt-level instruction to classify is a classification the model authors, and the model is the party ToolMaze identifies as over-trusting the corrupted output.

Specification drift has the cleanest detector of the five, and it is cheap enough that there is no reason to skip it.

The protocol gives you one contract-level check. Tools may also provide an output schema for validation of structured results; if one is provided, servers MUST provide structured results that conform to it and clients SHOULD validate structured results against it. The Security Considerations in the same revision put it plainly: clients SHOULD validate tool results before passing to the LLM.

{
  "name": "get_weather_data",
  "title": "Weather Data Retriever",
  "description": "Get current weather data for a location",
  "inputSchema": {
    "type": "object",
    "properties": {
      "location": { "type": "string", "description": "City name or zip code" }
    },
    "required": ["location"]
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "temperature": { "type": "number", "description": "Temperature in celsius" },
      "conditions": { "type": "string", "description": "Weather conditions description" },
      "humidity": { "type": "number", "description": "Humidity percentage" }
    },
    "required": ["temperature", "conditions", "humidity"]
  }
}

Validate against that and you catch a shape change and a type change. You catch nothing about a value that is well typed and wrong: temperature is a number whether it reads 22.5 Celsius or 22.5 Fahrenheit. Output schema validation is a specification-drift detector, not an output-drift detector. The schema is optional on the tool type, so for tools you own, make them declare one.

The alternative is being told, and the protocol defines a notification.

{
  "jsonrpc": "2.0",
  "method": "notifications/tools/list_changed"
}

Do not treat that as a safety net you get for free. At revision 2026-07-28, servers that declared the listChanged capability SHOULD send the notification to clients that have opened a subscriptions/listen stream with toolsListChanged set to true. It is a subscription, not a broadcast, so a client that reads tools/list once and never opens that stream is told nothing. An unconditional push to every client that saw the capability belongs to an earlier revision; what else changed on the wire at 2026-07-28 is covered separately.

The detector that does not depend on being told is a snapshot you take yourself, and the same revision supplies the guarantees that make one meaningful. The tool set returned from tools/list MUST NOT vary per-connection or as a side effect of other requests on the connection, though it MAY vary by the authorization presented on the request, and servers SHOULD return tools in a deterministic order. A stable set, a stable order and one documented axis of variation: that is a hashable object.

import hashlib
import json


def schema_fingerprint(tool):
    """Hash only the callable contract, not the prose around it."""
    payload = {
        "name": tool["name"],
        "inputSchema": tool.get("inputSchema"),
        "outputSchema": tool.get("outputSchema"),
    }
    blob = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(blob.encode("utf-8")).hexdigest()


def diff_inventory(previous, current):
    """previous and current map tool name to fingerprint, for ONE credential.

    Scope the snapshot per credential: the spec allows the tool set to vary by
    the authorization presented on the request, so a scope change would
    otherwise read as drift.
    """
    added = sorted(set(current) - set(previous))
    removed = sorted(set(previous) - set(current))
    changed = sorted(
        name for name in set(previous) & set(current)
        if previous[name] != current[name]
    )
    return {"added": added, "removed": removed, "changed": changed}

The authorization clause is why diff_inventory compares snapshots taken under one credential; compare across credentials and an entitlement change reads as drift.

A diff is not a decision. Classify it field by field against the fields your plan reads. Additive changes, a new optional field or a widened enum, route to continue with revalidation. Breaking changes, a removed required field, a narrowed type, a renamed parameter, a changed unit in a description, route to replan or stop, because every downstream node that consumed that field is planning against a contract that no longer exists. No published source sets a threshold where one becomes the other, so the line is one you write down per field.

Cross-source conflict is the second hardest family in ToolBench-X and the one where retrying is not merely useless but misleading. Both calls succeeded. Both returned well-formed data conforming to their schemas. They disagree, and a second identical call reproduces the disagreement exactly.

No standard, specification or paper says which one wins. ToolBench-X establishes only that the conflict is recoverable through cross-checking, which tells you the information is sufficient, not what to do with it. The precedence rule is something a human who knows the systems writes, per tool pair, before the agent runs.

Three decisions make a usable rule. Which source is authoritative for which field, and the answer may differ by field rather than by system, so "the ledger wins" is not a rule. How stale the non-authoritative source may be before its answer is discarded rather than reconciled, in the same units as the freshness stamp the tool returns. And what happens when both sit inside their windows and still disagree, which is where the stop branch earns its place: two systems of record that genuinely disagree is a data incident, and an agent that quietly picks one has destroyed the evidence.

Leave the choice to the model without a rule and it picks the more confident-sounding answer, because confidence in a tool response is a property of formatting rather than of provenance. Make the rule readable by the wrapper instead: it compares the two results, applies the precedence, and writes the outcome into the result the model reads with the losing value and the reason attached. That attachment is a hazard label, and the hint measurement puts a label far ahead of extra reasoning over an unlabelled conflict.

Three of the five hazards get more likely when the tools sit inside your own network, not less.

A public API versions its endpoints, publishes a changelog and deprecates on a schedule, because it has callers it cannot break. An internal claims system, a laboratory information management system or a core banking ledger has one caller and a database administrator. A view is altered, a nightly job changes its cut-off, a column moves from cents to the local currency unit, and there is no version string and no announcement. Specification drift and output drift are exactly the hazards that setup produces, and the fingerprint snapshot above is worth more there than anywhere.

Cross-source conflict can be built in by architecture. The read tool points at the reporting replica, because that is where analysts are permitted to query, while the write tool hits the transactional source, because that is where the record lives. Both are correct systems. Between the overnight batch and the moment the agent asks they disagree, and neither reports an error. Check that pair first in a regulated deployment.

What you get in exchange is the thing the hint measurement values at 25.5 to 35.5 points: you own both ends of the call. The tool-use documentation already gives the general form, telling you to write instructive error messages that say what went wrong and what the model should try next, with a rate limit and a retry-after delay as its worked example. That covers failures the tool admits to. The extension here is that the same discipline applies to results the tool considers successful.

Attach three fields to every successful result from an internal tool: the source identifier, the as-of timestamp of the data rather than of the response, and the freshness window the caller should treat as acceptable. A figure labelled as-of the overnight replica is one the agent can reason about; the same figure bare is one it will over-trust. That needs no model change and no new dependency, only fields in a response envelope you already control.

The measurement carries a data-boundary consequence too. Recovery rate needs trace rows containing tool outputs, and in a regulated deployment those outputs are the regulated data: the patient record, the claim, the position, the batch record. They do not leave for a hosted observability vendor. Compute the metric next to the orchestrator, from the log an auditor can read.

A published recovery rate tells you how a benchmark's models did against a benchmark's injected hazards, not how yours does against the tools it actually calls. The metric is reproducible from your own trace log: it is a conditional probability with a denominator you can populate.

from collections import Counter


def perturbation_recovery_rate(rows):
    """PRR = P(Recovered | Perturbation).

    Each row is one task attempt: perturbed says a hazard was observed or
    injected, recovered says the task still reached its expected end state.
    """
    perturbed = [row for row in rows if row["perturbed"]]
    if not perturbed:
        return None  # No hazard observed. PRR is undefined here, not 1.0.
    return sum(1 for row in perturbed if row["recovered"]) / len(perturbed)


def prr_by_hazard(rows):
    """Same metric split by hazard class, which is where the action is."""
    seen, recovered = Counter(), Counter()
    for row in rows:
        if not row["perturbed"]:
            continue
        seen[row["hazard"]] += 1
        recovered[row["hazard"]] += bool(row["recovered"])
    return {hazard: recovered[hazard] / seen[hazard] for hazard in seen}

The None guard is load-bearing. A run in which nothing was detected has an undefined recovery rate, not a perfect one, and a dashboard that renders 100 percent off an empty denominator will be the most reassuring chart in the building.

Then the limitation, because it changes how you read the number for the first few months. A hazard nothing detects produces no log line, so it never enters the denominator. Until the detectors above are installed, recovery rate measured on your traces is a lower bound on the hazards you are hitting and an upper bound on how well you recover. Watch the trend across weeks rather than the level, and expect it to fall the first time you install a detector, because you have started counting hazards you were previously eating.

Split it by hazard class, which is what prr_by_hazard is for. An aggregate hides what makes the split worth the effort: the ordering of the five hazard columns in ToolBench-X held across all twelve models. Pair the result with the repeated-run measurements in reliability lags accuracy in agent production.

This week. Take the tools your agent calls most and snapshot their contracts with schema_fingerprint, once per credential, into a file with a date on it. That file is worth nothing today and everything on the morning a required field quietly becomes optional. Then add two columns to your trace rows: a hazard class, null unless something detected one, and a boolean for whether the task reached its expected end state. Neither can be backfilled.