What are tokens in LLMs? Tokens and context windows explained

Tokens are the small chunks of text that a large language model (LLM) actually reads and writes: a token can be a whole word, part of a word, a single character or a byte. Every limit and price in an LLM API is counted in tokens, including the context window, which is the maximum number of tokens a model can handle in one request.

If you use Claude, ChatGPT, Gemini or a coding agent, tokens decide how much you can paste in, how long the answer can be and what it costs. This guide explains what tokens are, how to estimate them, how they relate to context windows and how to count them exactly.

What exactly is a token?

Models do not see letters or words. Before your text reaches the model, a tokenizer splits it into pieces from a fixed vocabulary and turns each piece into a number. The model works on those numbers and produces new ones, which are turned back into text.

Anthropic's glossary defines tokens as "the smallest individual units of a language model," which "can correspond to words, subwords, characters, or even bytes (in the case of Unicode)." It explains the trade-off behind the split: "Larger tokens improve data efficiency during inference and pretraining (and are used when possible), whereas smaller tokens allow a model to handle uncommon or never-before-seen words."

In practice that means common words are often a single token, while rare words, long identifiers and unusual strings are split into several.

Generation happens token by token too. NIST's Generative AI Profile describes it plainly: "LLMs predict the next token or word in a sentence or phrase."

How many words or characters is a token?

There is no universal ratio. Each model family has its own tokenizer, and the same text can produce different counts.

The Google figures come from the Gemini token counting docs; the Anthropic ones from its glossary and token counting guide. The Claude change is a good reminder that counts are model-specific. Anthropic's advice is to "Recount prompts against the model you plan to use rather than reusing counts measured against earlier models."

Ratios also shift with language. Anthropic notes that the number of characters per token "depends on the model and the language," so an English rule of thumb will not hold for every language.

Do images, PDFs and audio count as tokens?

Yes. Multimodal input is converted to tokens as well, and it adds up fast. Google's docs give concrete Gemini numbers:

• "Images ≤384 pixels in both dimensions count as 258 tokens." Larger images are tiled, and each tile counts as 258 tokens.
• Video: "263 tokens per second" for static processing.
• Audio: "32 tokens per second."

On the Claude side, a single request can include up to 600 images or PDF pages on models with a 1M-token window, and Anthropic warns you "might reach request size limits before the token limit."

What are input, output and thinking tokens?

APIs split usage into types, and they are priced differently.

• Input tokens: everything you send, including the system prompt, the conversation so far, tool definitions and files.
• Output tokens: everything the model writes back.
• Thinking tokens: on reasoning models, the internal reasoning before the answer. Anthropic's context windows guide says they "are billed as output tokens."
• Cached input tokens: repeated prefixes served from a prompt cache at a lower price.

For a concrete example, Anthropic's What's new in Claude Opus 5.5 page lists $4 per million input tokens and $20 per million output tokens, with cache reads at $0.20 per million (as of October 2026). Here output costs five times as much as input, so long answers and heavy reasoning can matter more to your bill than long prompts.

Anthropic's API reference explains why tokens are the unit: its API "bills and rate-limits by token counts, as tokens represent the underlying cost to our systems."

What is a context window?

The context window is the token budget for a single request. Anthropic defines it as "all the text a language model can reference when generating a response, including the response itself," and says it is "different from the large corpus of data the language model was trained on," acting instead as a "working memory" for the model.

Everything counts against it. In Anthropic's words: "Everything in the request counts toward the context window: the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions. The output Claude generates for the turn, including its extended thinking, counts too."

In a chat, the window fills turn by turn, because each new request resends the history. Anthropic describes this as "progressive token accumulation," where "previous turns are preserved completely."

Sizes vary by model. As of October 2026, Anthropic lists a 1M-token context window for current models such as Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5, with up to 128k output tokens per request, while some older models have 200k.

What happens when you run out of context?

Two things can go wrong.

Hard limit. If the input alone is larger than the window, Anthropic's API "returns a 400 invalid_request_error ("prompt is too long")." On Claude 4.5 and newer models, if generation then hits the limit, it stops with stop_reason: "model_context_window_exceeded".

Soft limit. Long before the hard limit, quality can slip. Anthropic warns that "more context isn't automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot." That is why context engineering focuses on giving the model the smallest useful set of tokens, and why RAG retrieves only relevant passages instead of pasting everything.

Coding agents manage this for you to a degree. Claude Code compacts long sessions and lets you check usage with /context; our Claude Code context window guide covers what fills it and how to keep it lean.

How do you count tokens?

Estimates are fine for planning. For exact numbers, ask the provider.

Claude. Anthropic's token counting endpoint "accepts the same structured list of inputs for creating a message, including support for system prompts, tools, images, and PDFs." It is "free to use but subject to requests per minute rate limits," and the result is "an estimate" that can differ slightly from actual usage.

response = client.messages.count_tokens(
    model="claude-opus-5-5",
    system="You are a scientist",
    messages=[{"role": "user", "content": "Hello, Claude"}],
)
print(response.json())  # {"input_tokens": 14}

Gemini. Google's docs give two methods: "Call count_tokens with the input of the request," or read the usage on the response, which reports input, output, thinking and cached tokens.

After the fact. Every Claude response has a usage field with input_tokens and output_tokens. Anthropic notes these "will not match one-to-one with the exact visible content," because requests are transformed before reaching the model.

How can you use fewer tokens?

• Cache stable prefixes. Put fixed instructions and documents first so prompt caching can reuse them. Note that Anthropic says cached prefixes "still occupy the context window"; caching changes the price, not the count. See Claude Code prompt caching.
• Keep system prompts tight. A system prompt is sent on every request.
• Retrieve, don't paste. Point an agent at a file instead of pasting its contents.
• Load instructions on demand. Agent Skills keep only a short description in context until a skill is used.
• Start fresh between unrelated tasks instead of carrying a long history.

Reusable prompts help here too. Kipt keeps your prompts and Agent Skills as Markdown in a private GitHub repo you own, so a tight, tested prompt is one Cmd+K search away instead of being retyped (and re-padded) each time.

What should you read next?

For what tokens cost in practice with a coding agent, see Claude Code cost. To understand how a model chooses each next token, read LLM temperature. And for the instructions that take up the first tokens of every request, see what is a system prompt.