Context window management

The concept

The context window is the material the model can consider during a turn. A larger window helps, but it does not remove the need to choose what belongs in the prompt. If you need the lower-level mechanics first, read tokens and context windows.

Long prompts fail in ordinary ways. The model may miss a detail, over-weight stale context, or spend attention on material that no longer matters.

The current approach

Put the important material close to the task. Use clear document boundaries and source names. Ask the model to cite or name the section it used when the answer depends on a source.

For long-running chats, summarize decisions and remove dead branches. Anthropic’s context-window guidance and OpenAI’s compaction guidance both treat context management as an active part of longer workflows.

For a task-level example, see long-document summarization or which model for long-context work.

A practical pattern

<decision_log>
- We chose Netlify static hosting.
- No backend, accounts, or database.
- Recipes are Astro content collection entries.
</decision_log>

<current_task>
Add a new recipe page that follows the existing recipe schema.
</current_task>

Use the decision log as constraints. Ignore earlier alternatives unless they explain
why a constraint exists.

This gives the model the current state without dragging the whole conversation along.

What to avoid

• Pasting every related document because the model has a large window.
• Asking “use all context” when only a few sections matter.
• Keeping old plans in the same prompt as the chosen plan.
• Hiding source names. A model can reason better when it knows which document a fact came from.