Tokens and context windows

What a token is

A token is the chunk of text a model reads and writes. It can be a whole word, part of a word, a punctuation mark, a space pattern, or a character from another language. Google’s Gemini token guide says a token is roughly four characters for Gemini models and gives 100 tokens as roughly 60 to 80 English words. That is a rule of thumb, not a portable conversion rate for every model.

Different model families split text differently. OpenAI’s tiktoken library exposes OpenAI tokenizers such as o200k_base and can look up the tokenizer for supported OpenAI models. Gemini, Claude, Llama, Mistral, and other models use their own tokenization paths. A prompt that counts as 500 tokens in one system may not count as 500 tokens in another.

For the workflow-level version of this topic, see context window management.

Why estimates vary

The visible text box is only part of the request. A real call may include a system prompt, developer instructions, retrieved documents, image or audio representations, tool definitions, tool results, cached content, safety wrappers, and the model’s answer. Some of that is visible to the person typing the prompt. Some of it belongs to the app or API wrapper.

Exact counts depend on that full request package. A rough text estimate can help with planning, but it stops being exact once the request includes provider wrappers, hidden instructions, attachments, tool definitions, multimodal inputs, or billing metadata. The Gemini token guide separates preflight counts from response usage. Response usage can include output, thinking, cached, and tool-use tokens after the interaction runs.

How the request grows

Conversation history counts too

In a multi-turn chat, whether it happens in a web product or through an API, the model answers from the context the product sends at that moment. A one-shot request starts with the current prompt package. A continuing chat usually includes retained history too. Anthropic’s context-window docs describe the input phase as previous conversation history plus the current user message. They also count system prompts, messages, tool results, images, documents, and tool definitions toward the window for Claude requests.

The practical effect is simple. If a 40-page document stays in the conversation, later turns may keep paying to carry it. Chat products can manage history differently, including rolling context or server-side compaction, but the work still has to fit into whatever context the model receives at generation time.

Long context is not free memory

A bigger context window gives the model more room, but it does not make everything in that room equally useful. Anthropic’s context-window docs discuss context rot, where accuracy and recall can degrade as token count grows. Extra context also increases latency and cost unless caching, summarization, or a product feature reduces what has to be processed again.

This matters most in long research chats and agentic coding sessions. Old tool output, outdated plans, repeated instructions, and stale documents can keep taking space after they stop helping. The fix is not to chase the largest window by default. Keep the useful context close and remove the rest when the task changes.

This is also why plan limits matter in free-tier vs paid-tier prompting.

How to estimate without fooling yourself

• Use the provider’s counter when the exact number matters. Claude has a token-counting API, and Gemini exposes count_tokens before a request plus usage metadata after a response. For OpenAI-only text, tiktoken is the right starting point.
• Count the whole request, not just the visible prompt. Include system instructions, tools, file content, retrieved context, and expected output. A short user message can still produce a large request.
• Treat multimodal counts as model-specific. Images, audio, video, and PDFs are converted into tokens or billable units by the selected platform. Google’s enterprise token docs note that multimodal preflight counts may be estimates and final billing usage is available in response metadata after execution.
• Start a new chat when the old context stops mattering. Long history is useful while the task is continuous. Once the task changes, old context becomes cost, latency, and sometimes confusion.