How to Structure the Context Window
The reason structure matters is the lost in the middle effect: models weight the start and end of a context more reliably than the center, so identical content produces different results at different positions. Content also needs to be distinguishable, the model has to tell an instruction from a retrieved document from a past message, and unlabeled concatenation blurs those lines. These steps produce a window whose layout puts important content where the model attends and makes every section legible.
Step 1: Pin the system instructions at the start
The system instructions define the model's role, its rules, and its output format, and they must never be displaced by accumulating content. Place them at the very top of the window, and where the model provides a dedicated system role, use it, because that role carries the strongest and most durable weight. Pinning means the assembly logic treats the instructions as fixed and protected, so a growing conversation or an oversized retrieval result can never push them out or bury them in the middle. When instructions drift out of their privileged position, the model starts ignoring its own rules, which is the most common structural failure and the easiest to prevent by treating the top of the window as reserved space.
Step 2: Group each source into a labeled section
Assemble the window as a set of distinct blocks, one per source, rather than a single run-on concatenation. Keep the retrieved knowledge in its own section, the long-term memory in another, the conversation history in another, and the tool results in another, each with a clear header or delimiter that names what it is. This grouping lets the model treat each kind of content appropriately, it can read retrieved passages as reference material, memory as established facts about the user, and history as prior dialogue, which it cannot do when everything is fused into one undifferentiated wall of text. Grouping is also what makes the window inspectable later, because you can see at a glance which section contributed what.
Step 3: Order sections by stability and priority
Arrange the sections so that stable framing sits near the start and volatile, high-priority content sits near the end. Instructions and durable background go first because they set the frame and benefit from the primacy of the top. The current user request and the most decision-relevant retrieved content go last, near the point of generation, because recency gives them the strongest influence on the answer. Lower-priority supporting material fills the middle, where the model's attention is thinnest and where it does the least harm if it is underweighted. This ordering deliberately aligns the model's attention gradient with your own ranking of what matters.
Step 4: Place the key content last
Identify the single most important item for the request, the one passage or fact the answer most depends on, and place it at the very end of the window just before generation begins. The recency zone is where the model's attention is strongest for the immediate task, so the item you most need used should occupy it. In a support system this might be the exact policy that governs the question, in a coding system the specific function to be edited, in a question answering system the passage that contains the answer. Restating the user's actual question at the end as well, after a long body of context, keeps the target of the response in the high-attention position and reduces the chance the model drifts.
Step 5: Delimit and label every section
Give every section an unambiguous boundary and a label, using consistent markers such as headers, tags, or a light structured format. The model needs to know where the retrieved documents end and the conversation history begins, and clear delimiters prevent it from reading one section's content as another's. Labeling also lets you state how each section should be treated, marking retrieved text as reference to be cited, or marking memory as established facts, which reduces context clash and confusion by making the status of each block explicit. Keep the delimiters consistent across requests so the model learns the layout and so your own tooling can parse the assembled window when you inspect it.
Step 6: Keep the structure fixed and testable
Use the same layout on every request so that the window's structure is a constant, not a variable. A fixed structure makes failures reproducible: when an answer is wrong, you know exactly where each kind of content sat, so you can tell whether the problem was placement, a missing section, or a bad item within a section. It also lets you write tests that assert the structure holds, that instructions are present and pinned, that no section exceeded its budget, that the key content landed last. A window assembled ad hoc, with a different shape each time, resists all of this, which is why a stable template is the foundation that makes the rest of context engineering debuggable.
Structure the window by pinning instructions at the top, grouping sources into labeled sections, ordering by stability and priority, placing the single most important item last, delimiting every section clearly, and keeping the layout fixed. This aligns the model's attention with your priorities and makes the window testable.
A Concrete Default Layout
A layout that works for most applications, and a good starting point to adapt, runs top to bottom as follows. First, the system instructions in the system role, pinned and protected. Second, durable background about the user or task drawn from long-term memory, framed as established facts. Third, the retrieved knowledge relevant to this request, in a labeled reference section. Fourth, a compressed summary of earlier conversation, if the session is long. Fifth, the recent conversation turns verbatim. Sixth, any tool definitions and their most recent results. Last, the current user request, restated cleanly, in the recency position. This ordering puts stable framing at the privileged start, fills the middle with supporting reference material, and reserves the high-attention end for the live request.
Adapt the layout to what the application depends on rather than following it rigidly. A retrieval-heavy assistant may promote the knowledge section closer to the end because it is the decisive content, while an agent may keep its running task summary near the end because that is what drives the next action. The principle is constant even as the specifics change: the start and end are premium positions, so put stable framing at the start and decisive content at the end, and let the middle hold what can afford less attention. Once you have a default layout that expresses these priorities, most tuning becomes small adjustments to it rather than reinvention on every request.
Where Structure Fits in the Pipeline
Structuring the window is the assembly stage of a larger pipeline, the point where selected and compressed content gets arranged into the final prompt. It comes after selection has chosen what to include and after compression has trimmed it to fit, and it comes before the observability step that logs what was sent. Treating it as a distinct, named stage rather than an incidental string-building step is what lets you reason about placement deliberately and change the layout without disturbing the stages that feed it. The full sequence, from naming sources through measuring results, is laid out in how to build a context pipeline, and structure is the stage where all the earlier decisions get their final position in the window the model actually sees.
Common Structuring Mistakes
A few structuring mistakes recur often enough to name, because each one undoes the benefit of good selection upstream. The first is letting the system instructions drift, appending new content on top of a window until the instructions slide out of their pinned position at the top and into the neglected middle, at which point the model begins to ignore its own rules. The fix is to treat the top of the window as reserved and protected, so no amount of accumulating content can displace the framing. The second is burying the decisive item, running excellent retrieval that surfaces exactly the right passage and then dropping it in the center of a long stack of supporting material where the model attends least, which wastes the retrieval win on a placement loss.
The third mistake is fusing sources into one undifferentiated block, concatenating instructions, retrieved documents, memory, and history with no delimiters, so the model cannot tell a rule from a reference from a past message and reads them all as one flat stream. This is the direct cause of the clash and confusion where retrieved content overrides instructions, and the fix is the labeled sections from the steps above. The fourth is changing the layout from request to request, assembling the window ad hoc so its shape is different every time, which makes failures irreproducible and defeats any attempt to test the structure. Each of these mistakes is a case of treating assembly as incidental string-building rather than as a deliberate, fixed stage, and avoiding them is mostly a matter of committing to one labeled, pinned, tested layout and enforcing it on every call.