Glossary

Token

A token is the unit a model actually processes text in - roughly a few characters or part of a word, not a whole word. Context windows, pricing, and output limits are all measured in tokens, not words or characters.

Context Window

The context window is the maximum amount of text (measured in tokens) a model can consider at once, including your prompt, any documents you provide, and its own response. Current frontier models support context windows up to roughly 1 million tokens.

System Prompt

A system prompt (or developer message, on some platforms) is a top-priority instruction channel set separately from the user's message, typically used to establish role, tone, and durable behavior for an entire conversation.

Zero-Shot Prompting

Zero-shot prompting means asking a model to perform a task with no examples included in the prompt, relying entirely on its instructions. Current reasoning models often perform this way by default without a meaningful accuracy loss.

Few-Shot Prompting

Few-shot prompting means including a small number of example input/output pairs in a prompt to steer format, tone, or structure. Current models pay close attention to example details, so a handful of diverse, well-chosen examples outperforms a large pile of similar ones.

<example>
Input: "The product broke after one day."
Output: {"sentiment": "negative", "urgency": "high"}
</example>
<example>
Input: "Works exactly as described, very happy."
Output: {"sentiment": "positive", "urgency": "low"}
</example>

Parameter Count

Parameter count is the rough number of learned values in a model, often shown as 7B, 12B, 31B, or similar. It is a useful size signal for memory and serving cost, but it is not a clean quality ranking because training data, architecture, tuning, context length, and tool support all matter.

Edge Model Size

An edge-sized model is built for constrained local hardware such as phones, laptops, browsers, or small devices. It trades maximum reasoning quality for lower latency, smaller memory use, offline availability, and lower serving cost.

Mixture of Experts (MoE)

Mixture of Experts is an architecture where only part of a larger model is active for a given token. A model can have a large total parameter count while using a smaller active subset per request, which can improve serving efficiency without making every token pay for the full model.

Quantization

Quantization reduces the precision used to store or run model weights, often so a model fits on cheaper or smaller hardware. It can lower memory use and improve speed, but aggressive quantization may reduce output quality, reasoning reliability, or multilingual behavior.

Model Routing

Model routing means choosing the right model version or product surface for a task before tuning the prompt. Current AI products often expose several models with different cost, speed, context, and capability tradeoffs.

Open-Weight Model

An open-weight model publishes downloadable model weights so developers can run, inspect, tune, or serve the model outside the provider's hosted chat product. The license, model card, and serving stack still determine what you can safely do with it.

Chain-of-Thought (CoT)

Chain-of-thought is a written, step-by-step reasoning trace, either produced by the model or explicitly requested in the prompt. On current reasoning models, manually requesting chain-of-thought ("think step by step") is usually unnecessary and can occasionally hurt output, since the model already reasons internally.

Old-style manual CoT: "Let's think step by step: first identify the cause, then consider two possible fixes, then pick one and explain why."

Current style: just ask the question directly - "What's causing this bug, and what's the best fix?" - and let the model reason internally without narrating each step.

Reasoning Effort

Reasoning effort is an API parameter (values typically range from none/minimal to low, medium, high, and sometimes xhigh or max) that controls how much internal reasoning a model performs before answering. It replaces older, manual chain-of-thought prompting as the way to control reasoning depth.

Adaptive Thinking

Adaptive thinking is Anthropic's term for Claude dynamically deciding when and how much to reason internally, calibrated by the effort parameter and the complexity of the query, rather than using a fixed manual reasoning budget.

Extended Thinking

Extended thinking is an older mechanism for giving a model a manual token budget for internal reasoning before it answers. On current Claude models this has been superseded by adaptive thinking with an effort parameter.

Thinking Budget / Thinking Level

A thinking budget (or the newer thinking_level parameter on Gemini) is the mechanism for controlling how much internal reasoning a model spends on a request, ranging from minimal reasoning for simple tasks to maximum depth for complex, multi-step problems.

Structured Outputs

Structured Outputs is a platform feature that constrains a model's response to match a defined schema (such as JSON with specific fields and types), enforced by the API rather than merely requested in prose.

Schema: { name: string, category: "billing" | "technical" | "other" }

Model returns exactly: {"name": "Jane Doe", "category": "billing"} - enforced by the API, not just requested in prose.

Schema-Constrained Output

A schema-constrained output feature makes the model return data that matches a declared schema, such as JSON with required fields, allowed enum values, and no extra properties. It is stronger than asking for "valid JSON" in prose because the API or serving layer participates in enforcing the shape.

Prompt-only request: "Return JSON with category and summary."

Schema-constrained request: declare category as one of ["billing", "technical", "account", "other"] and summary as a required string, then reject or repair outputs that do not match.

Prefill

Prefilling means providing a partial assistant response for a model to continue from, historically used to force output formats or skip preambles. Prefilled assistant responses are no longer supported on several current models - Structured Outputs and direct instructions have replaced this technique.

Old technique: seed the assistant turn with "Here is the JSON:\n{" so the model continues the object.

Current replacement: declare the schema directly via Structured Outputs, or instruct "respond with JSON matching this shape" - no partial assistant message needed.

Delimiters (XML Tags)

Delimiters are markers - XML-style tags, Markdown headers, or triple backticks - that separate distinct parts of a prompt (instructions, context, examples) so a model doesn't confuse one for another.

<instructions>Summarize the following document in three sentences.</instructions>
<document>{{long_text}}</document>

Without tags, a model has to guess where instructions end and the document begins.

Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation means supplementing a prompt with information retrieved from an external source (a search engine, database, or document set) rather than relying solely on what the model learned during training. Perplexity's core product is built around this pattern.

Chat Template

A chat template is the model-specific format that turns role-based messages into the raw token sequence a local or hosted chat model actually consumes. Using the wrong template can make an instruction-tuned model behave worse even when the visible prompt looks reasonable.

Current local-model practice: pass system/user/assistant messages through the tokenizer or serving framework chat template instead of manually concatenating labels like "System:" and "User:" unless the model card says that format is expected.

Tool Use / Function Calling

Tool use (also called function calling) lets a model invoke external functions or APIs - reading a file, running a search, calling a calculator - as part of producing its response, rather than answering from its own knowledge alone.

Agentic Workflow

An agentic workflow is a task where a model plans, takes actions via tools, observes the results, and adjusts its own next steps with minimal step-by-step human direction, as opposed to a single question-and-answer exchange.

Subagent

A subagent is a separate, delegated instance of a model (or a specialized tool built from one) that a primary agent hands off part of a task to, typically for parallel or isolated work that does not need to share the main context.

Hallucination

A hallucination is a confident-sounding but false or unsupported claim generated by a model. Explicitly permitting a model to express uncertainty, and grounding answers in retrieved sources, both reduce hallucination risk.

Prompt Injection

Prompt injection is an attack where untrusted content (a document, a web page, a tool result) contains instructions designed to hijack a model's behavior. It matters most in agentic and retrieval-based workflows where a model reads content it did not directly request from the user.