Sign up
ENRU
go2llm API documentation

context_length_exceeded and maximum context length exceeded: what to do

This error means the combined size of your request — system prompt, conversation history, and the room reserved for the reply — is bigger than the model can hold in one call. The ceiling belongs to the model, not to go2llm: the gateway doesn't keep a separate context-length limit on top of whatever the vendor already enforces, and it doesn't publish per-model window sizes in its own catalog. The request goes to the provider exactly as you sent it, and the provider is the one that says no.

Written 2026-09-26.

What the error looks like

OpenAI's Chat Completions API names this condition explicitly:

{
  "error": {
    "message": "This model's maximum context length is ... tokens. However, your messages resulted in ... tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

Anthropic's Messages API describes the same condition without a dedicated code — it's folded into the generic request-error type:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "prompt is too long: ... tokens > ... maximum"
  }
}

Both messages name two numbers — the model's ceiling and what your request actually added up to. This page deliberately leaves the numbers out: the exact ceiling is set per model by the vendor, it differs across models in the same family, and go2llm's own catalog (GET /v1/models) does not publish it as a field. Look it up on the model's own page in the vendor's docs before you plan around a number you read somewhere else.

go2llm's own error table has no dedicated code for this. 400 invalid_request, unsupported_content_type, model_protocol_mismatch, stateful_responses_unsupported, background_unsupported and idempotency_key_required cover everything the gateway itself checks before a request leaves the building — none of them are about token count. A context-length rejection is generated by the model provider after the request already cleared every gateway-side check, and it reaches you as a 400 with a generic go2llm-authored error.message; the vendor's own wording, when it survives the trip, travels separately in error.upstream_message, not in error.message itself. Whether the vendor's own wording and code survive the trip unchanged isn't something this page states as verified — producing one on purpose means sending hundreds of thousands of tokens at a real model, which nobody does just to screenshot an error. The safe assumption: check the HTTP status (400) and read error.upstream_message, when it's present, for words like "context", "length" or "token" rather than hard-coding a branch on error.code == "context_length_exceeded" alone.

What causes it, most common first

1. Conversation history grown without a trim

By far the most common cause. An app stores the whole chat and resends it in full on every turn. Each individual message looks reasonable; the running total after fifty replies doesn't. Agent frameworks make this worse and faster than plain chat: every tool call and every tool result gets appended to the history at each step, so a conversation that looks like ten user turns can carry a context many times larger underneath.

2. One large document pasted whole into a single turn

Extracted PDF text, a log file, a database dump — pasted into a message without trimming. This is the text-shaped sibling of pasting a binary file into a text endpoint (see 400 unsupported_content_type for the binary case); here the content is legitimately text, it's just too much of it.

3. max_tokens requested against a shared budget

On some APIs, max_tokens isn't a ceiling layered on top of the input — input and the requested output share one window. A prompt that would fit comfortably on its own can still be rejected because the client asked for a needlessly large max_tokens "just in case," leaving no room for the input that actually has to fit alongside it.

4. A framework or SDK inflating the prompt behind your back

RAG pipelines and agent SDKs like to inject retrieved documents, a tool schema, and boilerplate instructions into the system prompt automatically. The developer writes a three-sentence user question; the actual request sent to the model is several times longer, and nothing in the application code shows why.

5. A model swap mid-project, onto a smaller window

Not every model in a family shares the same context ceiling — the vendor sets it per model version, and a lighter or older sibling in the same lineup can carry a noticeably smaller one. A workflow built and tested against a large-window model can start failing the moment it's pointed at a cheaper or older model without re-checking that model's own limit.

How to test it and fix it, step by step

Step 1. Estimate the size of the request before you send it, not after it bounces. go2llm's own estimator is free — it neither reserves nor charges anything:

curl https://go2llm.tech/v1/messages/count_tokens \
  -H "x-api-key: $GO2LLM_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"..."}]}'

The response is {"input_tokens": N, "go2llm": {"exact": false, "method": "byte_estimate", "typical_error_pct": X}}. Read that carefully: it's a byte-based estimate using the same estimator go2llm uses to size its own balance reservation, not the model vendor's real tokenizer. It's good enough to tell you whether you're anywhere near a wall; it is not a promise that N equals what the model will actually count.

Step 2. If the estimate is large, trim client-side before sending rather than after the rejection:

def trim_history(messages, keep_last=20, system=None):
    """Keep only the most recent N turns; leave the system prompt alone."""
    tail = messages[-keep_last:]
    if system:
        return [{"role": "system", "content": system}, *tail]
    return tail

For long-running conversations, summarizing older turns with a cheap separate call beats a hard cutoff — it fixes the error and lowers the bill in the same move, since input tokens are billed whether or not the call succeeds.

Step 3. If one document is the culprit, extract only the relevant section before sending instead of the whole file — the same principle as decoding a PDF to text applies here, just with a length limit on top of it.

Step 4. Check max_tokens. On protocols where it shares a budget with the input, dropping it to what the task actually needs frees room:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.GO2LLM_API_KEY,
  baseURL: "https://go2llm.tech",
});

const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 512, // not 64000 "just in case"
  messages: [{ role: "user", content: "Summarize this in three bullet points." }],
});

How this differs from go2llm's own 413

go2llm's own hard limit is 413 request_too_large — the HTTP body over 20 MiB, checked in bytes, before the request goes anywhere near a model. It has nothing to do with tokens. For plain text without attachments, 20 MiB is an enormous number of tokens, so in practice a text-only request almost always hits the model's own token ceiling long before it would hit go2llm's byte ceiling. If you're seeing 413, the fix is body size; if the provider rejected the call with language about tokens and a maximum, the fix is context length, and there's nothing to raise on go2llm's side — the limit isn't ours to change. See 413 request_too_large for the byte-limit case.

Avoiding it next time

Measure tokens as part of the pipeline, not as a reaction to a failed call — count_tokens is free and doesn't touch your 20 RPM ceiling the way a real request would, since it never reserves or settles anything. Put a hard cap on conversation length and summarize instead of accumulating without limit. For work that genuinely needs a large window, check the specific model's own published ceiling with the vendor before committing to it — go2llm's catalog doesn't widen it, and it doesn't list it either.

Related errors

FAQ

Does go2llm trim or edit an oversized request for me? No. The gateway passes the request to the model as-is and doesn't touch its content. Trimming history or a document is entirely a client-side job.

Is there one context limit shared by every model on go2llm? No, and there's no code for it in the gateway's own error table either. The ceiling is set by the vendor per model, it isn't equal even within one model family, and go2llm doesn't publish it as a catalog field.

Does count_tokens count the same way the model will? No. It's a byte estimate — "exact": false, "method": "byte_estimate" — the same estimator the gateway uses to size a balance reservation, not the vendor's own tokenizer. It's the right tool for "am I close to the wall," not for counting to the exact token.

Does a context-length rejection cost anything? No. Nothing was generated, so there's nothing to settle — the same reserve-and-release mechanism that applies to every other rejected request applies here too. POST /v1/messages/count_tokens itself is free and never reserves anything in the first place.

Get a key and make your first request in 3 minutes → go2llm.tech. Per-model prices are on the pricing page, setup steps are in the docs.

API documentation Pricing and billing Support Referral terms (RU)