Context Engineering 101: Managing What the Model Sees

Prompt engineering taught everyone that the way you phrase a request to a language model changes the answer. Context engineering is the discipline that grew up around the realization that the phrasing is the easy part: the hard part is everything else in the model’s context window. Which instructions, which retrieved documents, which past messages, which examples, in what order, and trimmed how aggressively when the window fills up. Context engineering is the systematic design of everything the model sees before it generates, and for anything more complicated than a one-shot chat it is the single biggest lever on quality.

If that sounds abstract, the stakes are concrete. An agent that forgets its goal halfway through a long run, a RAG system that retrieves the right document but buries it under ten irrelevant ones, a chatbot that contradicts itself because its conversation history is too long to track: these are all context engineering failures, not model failures. The model is the same in each case. What changed is what it was given. Master context engineering and you stop blaming the model for problems you controlled all along.

Context engineering versus prompt engineering

The two are related but not the same, and the naming is a source of endless confusion. Prompt engineering is about the words of the instruction: the persona, the phrasing, the few-shot examples, the chain-of-thought nudge. Context engineering is about the full contents of the context window: the instruction is one ingredient, but so is the retrieved knowledge, the conversation history, the tool catalog, the system policy, and the output of the last action. Prompt engineering picks the recipe; context engineering manages the whole kitchen.

The reason context engineering became its own discipline is that models started being used for long, multi-step work. A single prompt lives in a small window and can be tuned by hand. An agent run accumulates tool results, retrieved passages, and reasoning over many steps, and someone has to decide what stays, what gets compressed, and what gets dropped. That decision-making is context engineering, and it does not reduce to clever phrasing.

The context window as a budget

The mental model that unlocks everything else is this: the context window is a fixed budget measured in tokens, and everything competes for space inside it. The system instruction costs tokens. Each retrieved document costs tokens. Every turn of conversation costs tokens. Every tool result, every reasoning trace, every few-shot example: all of it comes out of the same budget. When the budget is full, something has to give, and what gives determines whether the model stays coherent or degrades.

Context engineering is the practice of spending that budget well. It asks, for every token going into the window, whether it earns its place. High-value content (the user’s actual question, the single most relevant retrieved passage, the core instruction) should always make the cut. Low-value content (a verbose tool result that could be summarized, a stale conversation turn, a redundant example) should be compressed or removed. The goal is not to fill the window; it is to fill it with the highest-signal content available.

The six things that live in a context window

Most context windows hold some mix of six kinds of content, and context engineering is largely about managing each.

Instructions and policy. The system prompt that defines persona, rules, and boundaries. Usually short, always present, the highest-priority content.
The task or question. What the user actually wants. Must never be crowded out: losing the original goal mid-run is the classic context failure.
Retrieved knowledge. Documents fetched by RAG or facts pulled from memory. The largest and most variable component.
Conversation or action history. Prior turns, tool calls and their results, intermediate reasoning. Grows over the course of a run.
Tool definitions. The catalog of functions the model can call, with their schemas. Surprisingly large in tool-heavy agents.
Examples. Few-shot demonstrations of the desired output format or behavior. Powerful but token-expensive.

Context engineering is the ongoing negotiation among these six. When the window fills, you decide which retrieved passages to keep, which history turns to summarize, which examples to drop. These choices are invisible in a short chat and decisive in a long agent run.

Selection: choosing what enters

The first lever is selection: deciding what goes into the window at all. Not every retrieved document deserves a place; reranking and keeping only the top few is a context-engineering move as much as a retrieval move. Not every past turn needs to be kept verbatim; recent turns matter more than old ones for most tasks. Good selection is ruthless: it is better to include three highly relevant passages than ten mostly relevant ones, because the model’s attention is finite and diluting it with marginal content hurts the signal.

Selection has a recency and a relevance axis, and they often conflict. The most recent tool result is usually the most important for the next step, but the most relevant retrieved document might be one fetched five steps ago. Context engineering builds heuristics that balance these: always keep the original goal and the most recent observation, promote high-relevance items, and let go of the middle.

Compression: shrinking what stays

Summarization

When content must be kept conceptually but not verbatim, summarize it. A long tool result can become a one-line outcome. A ten-turn conversation can become a running summary that updates as new turns arrive. The summary costs far fewer tokens and preserves the information the model actually needs to act on. The risk is that summarization discards detail: if a later step needs an exact value that was summarized away, you have introduced a bug. The art is summarizing what is unlikely to be needed verbatim while keeping what might be referenced precisely.

Structured representations

Another form of compression is representing information more densely. A wall of prose describing a function’s output becomes a compact JSON object. A verbose error message becomes a code plus a short description. Structured representations carry the same information in fewer tokens and are easier for the model to act on, which is why well-designed tool outputs are themselves an act of context engineering.

Ordering: where things sit matters

The order of content in the window changes how the model weighs it. Two robust effects shape this. Models pay disproportionate attention to the start and the end of the context and less to the middle: the well-documented lost-in-the-middle effect. And they weight the most recent content more heavily, which is usually what you want but can cause them to drift from earlier, still-relevant instructions.

Practical context engineering uses these effects. Put the core instruction and the most important retrieved content at the very top. Put the current question or the most recent observation at the very end. Avoid stranding critical information in the middle of a long block. These are cheap rearrangements that materially change output quality, and they cost nothing once you know to make them.

Caching: not re-paying for the same tokens

A more mechanical but economically important lever is caching. Many model providers now cache the prefix of a context that does not change between calls (the system prompt, the tool catalog, a long retrieved document) and charge far less for the cached portion. Structuring the context so the stable parts come first and the changing parts come last lets you reuse the cache across turns. For a long agent run with a fixed system prompt and tool set, this can cut cost and latency dramatically. Context engineering includes ordering content to maximize cache hits, a concern that did not exist when every prompt was independent.

RAG and memory are context engineering

Two of the biggest patterns in applied AI are really context engineering in disguise. Retrieval-augmented generation is context engineering: it decides which external documents enter the window to ground the answer, and the quality of that selection is most of RAG quality. Long-term memory for agents is context engineering: it decides which past facts and episodes get recalled into the working window for the current step. Once you see RAG and memory as selection problems over the context, the whole field coheres: they are not separate techniques but the same technique applied to different content sources.

The failure modes of bad context

Goal drift. On long runs the original instruction gets pushed deep into the window and the model loses sight of it. Pin the goal at the top and re-inject it periodically.
Dilution. Too much retrieved content lowers the weight of each piece. Retrieve aggressively, keep only the best.
Stale instructions. A system rule from an earlier phase of a run no longer applies, but the model still follows it. Refresh or expire policy as the task progresses.
Lost detail. Over-summarization discards a value the model later needs. Summarize cautiously; keep anything that might be referenced exactly.
Cache breakage. Putting volatile content early in the window defeats prefix caching and raises cost. Stable content first, volatile last.

Common misconceptions

Context engineering is not just prompt engineering. It manages the full window, of which the instruction is one part.
A bigger context window does not solve context engineering. A larger budget postpones the problem; it does not remove the need to spend the budget well. Attention still dilutes.
More context is not better. The goal is the highest-signal context, not the most context.

Pro Tips
Put the goal first and the latest observation last. The two ends of the context get the most attention. Anchor the original instruction at the top and the current input at the bottom, and keep critical content out of the middle of long blocks.
Treat retrieval as context curation. In any RAG or memory system, you are choosing what enters the window. Rerank, trim, and prefer three precise passages over ten loose ones. The model’s attention is finite; spend it on signal.
Order for caching. Stable prefixes first, volatile suffixes last. It is a free win on cost and latency for any multi-turn system, and it costs nothing once you build the habit.