502 upstream_unavailable and 503: what's actually safe to retry

A 502 or 503 from an LLM API means your request never reached a model. The failure sits with the provider or the route, not your code. go2llm returns 502 upstream_unavailable with the message all safe upstream attempts failed, and 503 under the codes no_eligible_route or product_paused. All of these are safe to retry for plain text requests, but only up to the first byte of an SSE stream; after that, a retry duplicates the generation and the charge.

Written 2026-09-11.

What the error looks like

go2llm sends this once it has exhausted every safe attempt at a provider:

{
  "error": {
    "type": "upstream_unavailable",
    "code": "upstream_unavailable",
    "message": "all safe upstream attempts failed",
    "request_id": "gw_5e0c92a1d7b3486fa4e18c05b2d7f639"
  }
}

A second shape under the same error class fires when no route exists at all, status 503:

{
  "error": {
    "type": "upstream_unavailable",
    "code": "upstream_unavailable",
    "message": "no upstream route is currently available",
    "request_id": "gw_af71b3e2c5d8420e93a6f10d7c4b2e58"
  }
}

A third variant means routes exist but none of them qualify for this particular request: 503 with type no_eligible_route and the message no qualified compatible route is available. A fourth is a deliberate pause: 503 with type product_paused and a retry-after: 60 header.

The official Anthropic API answers the equivalent situation with a non-standard status, 529, and the type overloaded_error. That's Anthropic being overloaded, and it should be handled exactly like a 503. OpenAI's API returns a plain 503 with the code service_unavailable.

What causes it

1. The provider failed on every safe attempt

go2llm keeps more than one route to most models and, before a response starts, can fall back to a compatible route on the error classes it considers safe. all safe upstream attempts failed means those fallbacks are exhausted too. The problem is wider than one provider having a bad second.

2. No route exists for the model right now

no upstream route is currently available means the route to the requested model is temporarily gone. One key-free request tells you: GET https://go2llm.tech/status.

3. Routes exist, but none fit the request

no_eligible_route shows up when the available routes don't satisfy something about the request itself. The practical move is a same-protocol neighbor: claude-sonnet-5 instead of claude-opus-5 on POST /v1/messages, gpt-5.6-terra instead of gpt-6-astra on POST /v1/chat/completions.

4. The product is paused on purpose

product_paused means go2llm deliberately stopped serving that direction. The gateway sets retry-after: 60, so a retry before a minute is pointless.

5. The vendor itself is overloaded

Anthropic's API returns 529 overloaded_error during peak hours. That's not your request's fault and not go2llm's. It clears with waiting, or with a model switch if the overload runs long.

6. Your client timeout is shorter than the model actually takes

A 4,000-token reply from a large model can run tens of seconds. A 10-second HTTP client timeout severs the connection first, and your app sees a network error that looks like a 502 even though go2llm was still working the request. Set a client timeout of at least 120 seconds for non-streaming calls.

Fixing it, step by step

Step 1. Check GET https://go2llm.tech/status. It needs no key and shows the gateway's current state.

Step 2. Decide what's actually retryable. Safe: text requests to POST /v1/messages and POST /v1/chat/completions on 429, 502, 503, and network timeouts. Not safe, ever: a media request that returns ambiguous_upstream_result. The provider may already have accepted it, and a retry bills you twice. Also not safe: retrying an SSE stream after the first valid event. go2llm can't swap routes once a byte has gone out, so a client-side retry just produces a duplicate answer.

Step 3. Add backoff with a jitter and a ceiling:

import random, time
import httpx

RETRYABLE_STATUS = {429, 502, 503, 529}   # 529 is Anthropic's own overloaded_error
NEVER_RETRY_CODES = {"quota", "ambiguous_upstream_result"}

def call_text(payload, api_key, *, max_attempts=6, base=1.0, cap=32.0, timeout=120.0):
    """Retry a text request to go2llm on 429/502/503 with exponential backoff."""
    last = None
    with httpx.Client(timeout=timeout) as client:
        for attempt in range(max_attempts):
            try:
                r = client.post(
                    "https://go2llm.tech/v1/chat/completions",
                    headers={"Authorization": f"Bearer {api_key}"},
                    json=payload,
                )
            except (httpx.TimeoutException, httpx.TransportError) as exc:
                last, status, code, msg = exc, None, "network", str(exc)
            else:
                if r.status_code < 400:
                    return r.json()
                err = r.json().get("error", {})
                if not isinstance(err, dict):
                    err = {"message": str(err)}
                status = r.status_code
                code = err.get("code") or err.get("type")
                msg = err.get("message")
                last = RuntimeError(
                    f"{status} {code}: {msg} [request_id={err.get('request_id')}]"
                )
                if code in NEVER_RETRY_CODES or status not in RETRYABLE_STATUS:
                    raise last
                hint = r.headers.get("retry-after")
                if hint and hint.isdigit():
                    time.sleep(float(hint) * (0.5 + random.random() / 2))
                    continue

            if attempt == max_attempts - 1:
                raise last if isinstance(last, Exception) else RuntimeError(msg)
            delay = min(cap, base * 2 ** attempt)
            time.sleep(delay * (0.5 + random.random() / 2))   # jitter 50-100%

Six attempts with a 1-second base and a 32-second cap add up to roughly 47 seconds of total waiting in the worst case. Raise max_attempts to 8 and cap to 60 for background jobs; drop to 3 attempts and an 8-second cap for anything interactive.

Step 4. If a 502 keeps coming back past five minutes, send error.request_id (it starts with gw_) to support@go2llm.tech or Telegram @go2llm_support. go2llm keeps request logs for 90 days: route, usage, cost, status, and a sanitized error, but never the prompt or the response text.

Avoiding this later

Keep a same-protocol fallback model ready and switch to it after a second consecutive 502. On POST /v1/messages, claude-sonnet-5 is the fallback for claude-opus-5; on POST /v1/chat/completions, gpt-5.6-sol pairs with gpt-6-astra. Only the model string changes; the rest of the code stays the same.

Split your timeouts: 120 seconds for a non-streaming call, and a separate, longer one for a stream. A timeout that's too short manufactures 502s in your own metrics and hides whether the problem is the provider or your client.

Don't retry everything. 400, 401, 402, 403, 404, and 409 return the same answer on a second try and just burn your 20-requests-per-minute ceiling. Branch on error.code, not the HTTP status. The same 429 covers a retryable rate_limit and a non-retryable quota.

None of this costs you anything to test. A failed go2llm request charges nothing. The gateway reserves an estimate before the call and releases it if the call fails, so the balance never drops for a request that didn't complete. The real cost of a successful request lands in usage.cost_usd, including in the final block of an SSE stream.

Related errors

• 429 rate_limit_error — same backoff, a different cause.
• The full error reference and API error docs.

FAQ

What's the actual difference between 502 and 503 on go2llm? 502 upstream_unavailable with all safe upstream attempts failed means go2llm tried providers and every attempt failed. 503 means no suitable route existed in the first place: no upstream route is currently available, no_eligible_route, or a deliberate product_paused with retry-after: 60.

Can I retry a stream that cut off mid-response? Not if you've already received at least one valid SSE event. go2llm can only pick a fallback route before the first byte goes out; after that, a client retry produces a duplicate generation and a second charge.

What do I do with ambiguous_upstream_result? Don't retry it automatically. It means the provider may have already accepted a media request, and an automatic retry risks paying for it twice. Check the result by hand, and if you do send it again, use a new Idempotency-Key.

Anthropic is returning 529. Is that go2llm's fault? No. 529 overloaded_error is Anthropic's own non-standard code for being overloaded on their side. Handle it in the same branch as a 503: exponential backoff with jitter, and a model switch if the overload doesn't clear.

Get a key and make your first request in 3 minutes → go2llm.tech. Registration credits $1 to your balance. Per-model prices are on the pricing page.