Sign up
ENRU
go2llm API documentation

500 internal server error on the OpenAI API: what reaches you through go2llm

A 500 from a model provider's API means something broke on their side while handling a request that was, as far as they could tell, perfectly fine to accept. It isn't your JSON, your key, or your balance — the vendor's own infrastructure hit a bug or a crash partway through. That also makes it the one class of error where "just retry it" is usually the right first move, not a cop-out.

Calling the same model through go2llm, which shape you see depends on whether go2llm's provider router is switched on for the gateway you're hitting. With the router on, it makes exactly one attempt per request and relays whatever status the provider answered with — 500 included — wrapped in go2llm's own envelope with error.code set to upstream_http_500. With the router off, the older code path retries a 500 across other eligible routes first, and only reports 502 upstream_unavailable once every one of those has also failed. Check which mode is live right now with no key required — curl -s https://go2llm.tech/status and read the providerRouterV1 field; it's true on go2llm.tech today, but that's a live setting, not a permanent default, so verify it rather than assuming. Either way the underlying cause is identical; the difference is only whether you see the exact number or a generic 502. The one true gap: an unhandled exception inside the gateway's own request handling, which is a gateway bug rather than a relayed upstream failure — that one always surfaces as a literal 500, but with a bare {"error": "..."} body instead of the usual error.type/error.code/error.request_id envelope, which is how you tell the two apart.

Written 2026-10-04.

What each one looks like

OpenAI's own API, when its infrastructure fails on a request it otherwise accepted:

{
  "error": {
    "message": "The server had an error while processing your request. Sorry about that!",
    "type": "server_error",
    "param": null,
    "code": null
  }
}

go2llm, for the same underlying condition, with the provider router on — the status stays 500, and only the envelope changes:

{
  "error": {
    "type": "upstream_error",
    "code": "upstream_http_500",
    "message": "The upstream model is temporarily unavailable. Retry the request (request_id: gw_5e0c92a1d7b3486fa4e18c05b2d7f639).",
    "request_id": "gw_5e0c92a1d7b3486fa4e18c05b2d7f639"
  }
}

On the older, non-router path — used when the provider router is switched off — a 500 is retried across other eligible routes first, and only turns into a 502 once all of those retries are also exhausted:

{
  "error": {
    "type": "upstream_unavailable",
    "code": "upstream_unavailable",
    "message": "all safe upstream attempts failed",
    "request_id": "gw_5e0c92a1d7b3486fa4e18c05b2d7f639"
  }
}

If you're using the OpenAI SDK pointed at go2llm's OpenAI-compatible endpoint, the router-path response above arrives as the SDK's own APIError (or InternalServerError, depending on SDK version) with status_code set to the literal 500, not 502 — worth knowing if any of your error handling branches on the exact number rather than catching the general "server-side failure" class. Only the older, non-router fold-in still shows up as 502.

What causes it, most common first

1. A transient fault on the vendor's own infrastructure

The single most common cause, and it's explicitly not about your request. It's the same category of problem as Anthropic's 529 overloaded_error, just phrased as an outright failure rather than "too busy" — see 529 overloaded_error for that variant. Both clear on their own, usually within seconds to minutes.

2. An unusual but technically valid request triggers a bug on the vendor's path

Deeply nested tool schemas, an uncommon combination of parameters, or a content structure that's valid by the spec but rarely exercised can occasionally trip a bug in the vendor's own request handling rather than being caught cleanly as a 400. This is far less common than plain infrastructure trouble, and it's the one case where changing the request — not just retrying it — actually helps.

3. A request large enough to exhaust resources before validation catches it

A request with very large tool definitions or an unusually long prompt can, in rare cases, hit a resource limit on the vendor's side that should have produced a clean 400 but instead surfaces as a 500, because the failure happens after the point where the vendor's own input validation would normally have caught it.

4. A genuinely bad period for the provider, affecting many customers at once

Repeated 500s for the same model across an extended window, not tied to any specific request shape, usually means the provider itself is having a rough stretch — the same root cause a status page would call an incident. This isn't specific to you and doesn't respond to changing the request.

Testing which one you're looking at

Check go2llm's own status first, no key required — the providerRouterV1 field tells you which envelope shape to expect: true means a literal status with upstream_http_5xx, false means the older path's generic 502 upstream_unavailable.

curl -s https://go2llm.tech/status

Then isolate whether it's your payload or the provider. Retry a minimal, known-good request for the same model:

curl -s -o /dev/null -w "%{http_code}\n" https://go2llm.tech/v1/chat/completions \
  -H "Authorization: Bearer $GO2LLM_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"gpt-5.6-luna","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'

If the trivial payload also fails, the provider is having a bad moment and it isn't about your specific request. If only your original, more complex payload fails while the minimal one succeeds, something about your request's shape is the actual trigger — start simplifying it (drop tool definitions, shorten the prompt, remove uncommon parameters) to find which part.

Fixing it

Retry with backoff. This is the one error class where an immediate second attempt frequently just works, because the failure is almost never deterministic. Branch on the status range (>= 500), not a single code — a client has no way to know from the outside whether the router is on for this request — and treat any error.code that starts with upstream_http_, or equals upstream_unavailable, as the same upstream failure. Then reuse the same retry loop as any other upstream failure; see 502 upstream_unavailable for a full jittered backoff implementation, since the handling past that point is identical.

import httpx

def call_once(payload: dict, api_key: str) -> httpx.Response:
    return httpx.post(
        "https://go2llm.tech/v1/chat/completions",
        headers={"Authorization": f"Bearer {api_key}"},
        json=payload,
        timeout=120.0,
    )

r = call_once(payload, api_key)
if r.status_code >= 500:
    err = r.json().get("error", {})
    code = err.get("code", "")
    is_upstream_failure = code.startswith("upstream_http_") or code == "upstream_unavailable"
    print(f"upstream failure, code={code}, matched={is_upstream_failure}, request_id={err.get('request_id')}")
    # retry with backoff here — see the 502/503 page for the full loop

If it's request-shaped, simplify first. Strip tool definitions down to the minimum needed, shorten an unusually long prompt, and reintroduce pieces one at a time until the failure comes back — that identifies the actual trigger instead of guessing.

If it persists past a few minutes across retries, it's a provider-side incident rather than something a retry loop will fix. Switching to a same-protocol sibling model is the practical move while it clears.

Avoiding it later

Keep a same-protocol fallback model ready, exactly as for any other 502 — a second consecutive failure on the primary model is a reasonable trigger to switch rather than keep hammering the same route. Favor simpler, well-tested request shapes (standard tool schemas, ordinary content blocks) where the task allows it; the rarest failures in this category come from edge cases in how a request is built, not from what it asks the model to do.

None of this costs anything to investigate. A request that fails with a 502 was never billed — the reservation made before the call is released, and the real cost of a request only lands in usage.cost_usd when a response actually comes back.

Related errors

FAQ

Will go2llm ever return a literal 500 to me? Yes, routinely — with the provider router on (check providerRouterV1 at /status; it's true on go2llm.tech today), a provider's own 500 passes through as a literal 500, wrapped in go2llm's envelope with error.code: "upstream_http_500". It only gets folded into 502 upstream_unavailable on the older, non-router path, once every fallback route has also failed. A gateway-side bug producing an unhandled exception also surfaces as a literal 500, but you can tell the two apart from the body: a relayed upstream failure carries the usual error.type/error.code/error.request_id envelope, a gateway bug doesn't.

Is a 500-class failure safe to retry? Yes, more so than almost any other error class. It's rarely tied to something specific about your request, so an immediate retry with backoff frequently succeeds on the next attempt.

How is this different from a 504 gateway timeout? A 500-class failure means the provider answered with an explicit failure. A timeout means it never answered in time at all. On the router path, that difference still shows: a 500 passes through as upstream_http_500, while go2llm's own deadline elapsing stays a 502 with error.code: "timeout" — see the dedicated 504 gateway timeout page for that case.

Does a failed request affect my balance? No. The cost estimate reserved before the call is released when the call fails, and nothing is charged unless a response actually comes back.

Get a key and make your first request in 3 minutes → go2llm.tech. Per-model prices are on the pricing page, setup steps are in the docs.

API documentation Pricing and billing Support Referral terms (RU)