Anthropic API Error 529 Overloaded: Agent Retry Policy

An agent loop does not need eleven handlers. It needs four answers per code.

The body shape is the same for every code, and the field that matters most later is a sibling of the error object rather than a member of it.

{
  "type": "error",
  "error": {
    "type": "not_found_error",
    "message": "The requested resource could not be found."
  },
  "request_id": "req_011CSHoEeqs5C35K2UUqR7Fy"
}

Three details there are load-bearing. The 500 bullet is the only one in Anthropic's reference that prescribes exponential backoff, and it also tells you to contact support with the request ID if the error persists. The 504 bullet does not say retry, it says consider using the streaming Messages API for long-running requests, which is a design change. And there is no 503 anywhere in the table, nor 502, nor 522, which is the whole of the gateway section below. Everything in the "no" column belongs to the class of failures a wrapper must stop retrying immediately, the same argument as the deterministic 400 class, where a retry is the wrong reflex.

The 429 bullet in the error reference names three causes in one sentence: your organization has hit a rate limit, reached its usage tier's monthly spend cap, or reached a spend limit on the Claude Code workspace. All three return rate_limit_error. Two of them clear on a timer. The third does not clear for the rest of the month.

The discriminator is not the header. It is error.details.error_code, and the documentation names it explicitly as the field that tells a spend-cap response apart from a rate limit. Branch on that token, not on whether retry-after happened to be present.

{
  "type": "error",
  "error": {
    "type": "rate_limit_error",
    "message": "You have reached your API usage limits: your organization has crossed its monthly API usage threshold, set based on your organization's API tier. You will regain access on 2026-09-01 at 00:00 UTC.",
    "details": { "error_code": "enforced_spend_limit_reached" }
  },
  "request_id": "req_018EeWyXxfu5pfWkrYcMdjWG"
}

That response has no retry-after header, and the documentation is blunt about the consequence: retrying, including the SDKs' automatic retries, fails until access resumes, which is 00:00 UTC on the first day of the next month unless you request a higher limit sooner. An agent that treats this as a transient 429 burns its entire retry budget on every turn, for days, and logs nothing that separates the pattern from a busy afternoon.

The fourth outcome is not a 429 at all. A spend limit you set yourself returns HTTP 400 with type invalid_request_error, and the message begins "You have reached your specified API usage limits" (or "You have reached your specified workspace API usage limits" for a workspace limit). A wrapper that classifies every 400 as a permanent code defect reports budget exhaustion as a malformed request. The exception to that exception: limits on the Claude Code workspace can return a 429 instead.

Four rows, two status codes, two type strings. A self-set limit cannot exceed your current tier's cap, so a wrapper carries both branches: the 400 fires first if you set a limit, and the tier-cap 429 is what you hit if you did not.

Anthropic's own documentation states the behaviour you inherit: the official SDKs automatically retry transient failures such as connection errors, rate limits and 5xx server errors, with exponential backoff, twice by default, honouring the retry-after header when present, and each client accepts a maximum-retries option to configure or disable it.

In the Python SDK source the constants are DEFAULT_MAX_RETRIES = 2, INITIAL_RETRY_DELAY = 0.5 and MAX_RETRY_DELAY = 8.0, with a default request timeout of 10 minutes and a 5.0 second connect timeout. The sleep is min(0.5 * 2**n, 8.0) scaled by a jitter factor of 1 - 0.25 * random(), so every sleep lands between 75 and 100 percent of nominal. With the default of 2, the exponent takes the values 0 and 1, so the nominal sequence is 0.5s then 1.0s: 1.5 seconds of total backoff, or between 1.125 and 1.5 seconds after jitter.

Two details matter before you rely on it. The Python SDK does not special-case 529: its retry predicate covers 408, 409, 429 and any status at or above 500, with a non-standard x-should-retry header of true or false overriding all of them, and 529 qualifies purely because it is at or above 500. And both SDKs parse the same three forms, retry-after-ms first, then retry-after as seconds, then an HTTP-date; Python alone gates the result, keeping it only when it is above 0 and no more than 60 seconds. A longer value is discarded and the client falls back to the ladder, which caps at 8.0 seconds, so it retries far earlier than the server asked. TypeScript sleeps for whatever it parsed.

So the question is not whether to retry but what wall-clock budget you buy when you raise the number.

Those figures are nominal, exclude the request time itself, and get scaled down by up to 25 percent by jitter. There is no correct row, only a budget: an agent turn that can tolerate a minute of dead time picks a high row, an interactive one does not, and a run of N turns multiplies whichever row you pick. Decide it at the loop rather than inheriting 1.5 seconds by accident, and pair it with a stop condition, because a retry ladder that never terminates is the same defect as a tool loop that is not making progress.

Here is the failure that survives a careful retry wrapper. The documentation states plainly that on a streaming response over server-sent events, an error can occur after the API returns a 200, and error handling then does not follow the standard mechanisms. The streaming reference then gives its example of such an event, and the example is the overload itself: during periods of high usage you may receive an overloaded_error, which would normally correspond to an HTTP 529 in a non-streaming context.

event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

The Python SDK's SSE iterator skips ping events and, when it sees sse.event == "error", calls the client's status-error factory with the parsed body and the response object. That factory dispatches purely on response.status_code, with branches for 400, 401, 403, 404, 409, 413, 422, 429 and 529 and then a catch-all for anything at or above 500. A streaming response's status code is 200. None of those branches match, so the call falls through to a bare APIStatusError carrying status 200, and anthropic.OverloadedError is never constructed.

import anthropic

# Turn the SDK's own retries off; the loop above owns the policy.
client = anthropic.Anthropic(max_retries=0)

RETRYABLE_TYPES = {"overloaded_error", "api_error", "timeout_error"}

def classify(exc: anthropic.APIStatusError) -> str:
    body = exc.body if isinstance(exc.body, dict) else {}
    err = body.get("error") or {}
    err_type = err.get("type")
    err_code = (err.get("details") or {}).get("error_code")

    if err_code == "enforced_spend_limit_reached":
        return "stop_run_spend_cap"
    if err_type == "rate_limit_error":
        return "wait_and_retry"
    if err_type in RETRYABLE_TYPES:
        return "backoff_and_retry"
    if err_type == "invalid_request_error":
        # Also fires on an org or workspace spend limit you set.
        return "stop_run_check_message"
    return "stop_run"

def retry_after_seconds(exc: anthropic.APIStatusError) -> float | None:
    value = exc.response.headers.get("retry-after")
    return float(value) if value else None

Nothing in classify reads a status code, which is the point: the body carries the truth on both the 529 path and the 200 path, the status on only one. The same dispatch quirk explains two exported classes you should not rely on. ServiceUnavailableError and DeadlineExceededError exist, but the dispatcher has no 503 or 504 branch, so a 503 or 504 from the direct API path raises InternalServerError through the catch-all.

# Non-streaming: a 529 raises anthropic.OverloadedError (status_code 529).
# Streaming: the same overload arrives as an SSE error event on a 200
# response, and the SDK raises it through the same status dispatcher,
# which has no branch for 200, so you get a bare APIStatusError.
try:
    with client.messages.stream(
        model="claude-sonnet-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "..."}],
    ) as stream:
        message = stream.get_final_message()
except anthropic.OverloadedError:
    action = "backoff_and_retry"          # never reached mid-stream
except anthropic.APIStatusError as exc:
    action = classify(exc)                # this is the branch that fires

A mid-stream overload also means a partial turn: some content blocks arrived, the call did not complete, and any tool call already dispatched may or may not have landed. Retrying blind is how you double-write. Hand off to the postcondition and idempotency machinery in verifying the turn actually happened before you retry it rather than reimplementing it here.

The moment a proxy sits between the agent and the weights, the error contract becomes the proxy's. An open issue on the LiteLLM tracker, #36655, filed 12 August 2026 against v1.96.0 and labelled for the proxy, the LLM translation layer and Claude Code, reports that the Anthropic-protocol route returns OpenAI-shaped errors and that an upstream capacity 503 from Bedrock therefore never becomes overloaded_error.

503
{"error": {"message": "{\"message\":\"Bedrock is unable to process your request.\"}",
           "type": "None", "param": "None", "code": "503"}}

Look at what a client has to work with. The envelope is {"error": {...}} with message, type, param and code, which is not the Anthropic shape. The upstream message is doubly encoded, a JSON string inside a JSON string. type and param are the string literal "None", which is the fallback the proxy's exception handler hands to getattr when the exception carries no such attribute, and the issue quotes that handler as raising a ProxyException that the project's own type definitions describe as mapping exactly to OpenAI exceptions. The reporter's reading of the client is that overload is detected on exactly two conditions, a status equal to 529 or a message containing the literal "type":"overloaded_error", so an OpenAI-enveloped 503 matches neither and the user sees a generic API error. That is the reporter's reading rather than a documented client contract, so test it against your own client.

The correct path in the same tree does not rescue you either. LiteLLM's Anthropic error-mapping module defines a status-to-type map covering 400, 401, 403, 404, 413, 429, 500 and 529, with 402, 409 and 504 absent and no 503 entry, and its lookup falls back to api_error for anything unmapped. So even a 503 routed through the mapper arrives as api_error. The machinery to rebuild an Anthropic-shaped envelope exists in that file; the reported defect is that the generic exception branch on the /v1/messages route does not call it, and the route's own retry helper does not retry 5xx, so the 503 reaches that handler on the first attempt.

The general rule falls out of the error table: no 503 exists in Anthropic's published status set, so a 503 reaching a client that speaks the Anthropic protocol is positive evidence that something between you and the model produced it. Treat that as a signal, not noise: log the status and the envelope shape separately, and normalise upstream statuses into Anthropic type strings once at the proxy rather than in every client. Which gateway you run is a separate decision, argued in choosing the gateway in the first place.

Move the weights inside your perimeter and the same defect reappears one hop closer to the model. vLLM has admission control, exposed as two scheduler fields, max_num_queued_reqs and max_num_queued_tokens, both declared with a default of None, meaning no limit. An unconfigured private endpoint therefore has an unbounded request queue, and the saturation that would be a 529 on the hosted API shows up instead as time-to-first-token creeping up, which no error-driven retry policy can see. The dashboard is green, the queue is deep, and every agent turn is waiting.

Turning the caps on is one change to the serve command. The values below are illustrative; size them from your own measured queue depth and prefill backlog.

vllm serve <model> \
  --max-num-queued-reqs 256 \
  --max-num-queued-tokens 32k

With max_num_queued_reqs set, the engine rejects a request when unfinished requests, waiting plus running, plus the slots this request needs would exceed the cap. Rejections raise QueueOverflowError or MaxQueuedTokensError, and both carry HTTPStatus.SERVICE_UNAVAILABLE, that is HTTP 503. The error responder sets the type string to the HTTP status phrase, and vLLM's Anthropic-protocol router copies that string straight through into an Anthropic-shaped error model whose type field is a plain string rather than a constrained enum. Composed from those code paths rather than captured from a live server, the body looks like this.

{
  "type": "error",
  "error": {
    "type": "Service Unavailable",
    "message": "The engine is currently busy and cannot accept new requests. Please try again later or on a different instance."
  }
}

Anthropic-shaped envelope, non-Anthropic type string, and no request_id field, because vLLM's Anthropic error response model has exactly two fields and that is not one of them. An agent written against the hosted API and moved inside the perimeter loses three things at once: the status code its overload branch tests, the type string its body branch tests, and the identifier its log line records.

For a regulated deployment the error contract is a conformance surface you own and have to test. Write an assertion for the exact body your gateway emits under upstream saturation, run it in CI against a stub, and put queue depth and time-to-first-token on the dashboard beside error rate.

Anthropic rate-limits with a token bucket that replenishes continuously up to your maximum rather than resetting at fixed intervals. Two consequences argue against solving saturation with more retries.

The first is granularity. The documentation warns that you might hit rate limits over shorter intervals: a limit of 60 requests per minute might be enforced as 1 request per second, and short bursts can exceed the limit. A fan-out that releases twelve subagents in the same instant is measured against that shorter window, not against the per-minute number it sits inside.

The second is ramp. Both the errors page and the rate-limits page note that a sharp increase in an organization's usage might produce 429s because of acceleration limits, and the errors page hedges it as happening in rare cases. The published mitigation is to ramp traffic up gradually and maintain consistent usage patterns. A cron-scheduled batch of runs that all start on the same minute is the opposite of that, and so is a deploy that raises subagent concurrency from two to twenty.

So put the control above the SDK, not inside it. Three pieces, in this order:

Moving a scheduled fan-out to the Message Batches API changes which bucket it drains: batches have their own requests-per-minute limit plus a separate cap on batch requests in the processing queue, distinct from the Messages API limits. For work that does not need a synchronous answer, that is a cheaper fix than any retry tuning.

The mirror-image problem, designing the retry-after and rate-limit headers you emit from your own services, is covered separately, and the two designs should agree.

Every API response includes a unique request-id header carrying a value like req_018EeWyXxfu5pfWkrYcMdjWG, and the same identifier appears as the request_id field in error response bodies. The Python and TypeScript SDKs expose it as a _request_id property on top-level response objects; the C#, Go, Java and PHP SDKs expose it through raw-response accessors, and Ruby through middleware. On Claude Platform on AWS there are two identifiers on the same response, the AWS one in x-amzn-requestid and the Anthropic one in request-id, and a log line that records only one of them is unusable for half the support paths.

curl -sS -D - -o /dev/null https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello, Claude"}]
  }'

That dumps the response headers and discards the body, which is what you want while wiring the logging. Two families are on it. request-id is the identifier you attach to the attempt. The anthropic-ratelimit-input-tokens-remaining, anthropic-ratelimit-output-tokens-remaining and anthropic-ratelimit-requests-remaining headers, with their matching -limit and -reset companions, show how close the run is to the wall before it hits it; the -reset values are RFC 3339 timestamps and the token-remaining values are rounded to the nearest thousand.

Record, per attempt and not per run: the request id, the error type, error.details.error_code where present, the attempt number, the retry-after value you honoured, and the remaining-capacity headers. Then aggregate retries per run. A run that succeeded on its tenth attempt spent nine retries and 47.5 seconds of nominal sleep getting there. Under per-run success logging it is indistinguishable from a run that succeeded first time, and the first sign of trouble is a ticket about latency. Under per-attempt logging it is a capacity incident with a start time, a request id per attempt, and a header series showing the headroom draining.

Three changes this week, in order of what they buy per hour spent. Move your error branch off the status code and onto the response body, so error.details.error_code separates the spend cap from the throughput limit and a mid-stream overloaded_error on a 200 still routes to the overload path. Pick a max_retries from the wall-clock table instead of inheriting 1.5 seconds. Then add the request id and the attempt number to the log line, run one day of traffic, and count how many green runs took more than one attempt. If you serve the model from your own hardware, add one more: send a request to your endpoint with the queue full and write down exactly what comes back, because that body, not Anthropic's, is the contract your client has to satisfy. More on this in our AI agents guides.