The Lost in the Middle Problem in Long Contexts

The lost in the middle effect was named by research that tested how well models retrieve a single relevant fact placed at different positions inside a long context. The finding was a U-shaped accuracy curve: models answered most reliably when the needed fact was near the start or the end of the context, and least reliably when it sat in the middle. The dip in the center was large enough to matter in practice, a fact that would be used correctly at the edges could be effectively invisible in the middle of a long enough window. The effect has held across model families and has persisted even as context windows have grown, which is why it remains a core concern of context engineering rather than a quirk of one early model.

Why It Happens

The effect comes from how attention behaves over long sequences. A model's attention is not uniform across the window, it is shaped by training and by architecture in ways that privilege the boundaries. The beginning of the context tends to hold the system instructions and the framing of the task, which the model is trained to weight heavily, and there is a general primacy pull toward the earliest tokens. The end of the context is the most recent input, immediately adjacent to where the model begins generating, and recency gives it strong influence over the next tokens. The middle has neither advantage, it is far from the framing and far from the point of generation, so content there competes for a thinner slice of attention.

This connects directly to context rot, the broader decline in quality as a window fills. Lost in the middle is one of the specific mechanisms behind rot: as a window grows, more content is pushed into the low-attention middle zone, so the fraction of the window the model actually attends to well shrinks. A short window has little middle to get lost in, which is part of why lean windows outperform full ones. The two ideas reinforce each other, rot says longer windows degrade, and lost in the middle explains one reason why, the center of a long window is where attention is weakest.

What It Means for Context Engineering

The immediate lesson is that assembling a window is not just about which items to include, it is about where they go. If your retrieval step surfaces the one document that answers the question but your assembly logic drops it in the middle of a long stack of supporting material, you can get a wrong answer from a window that contained the right information all along. This is a failure that never shows up if you only look at whether the relevant content was retrieved, because it was, the failure is in placement, and it is only visible if you consider position as part of the design.

The lesson compounds in systems where the window grows over time. A long conversation accumulates turns, and without management the important early context, including the system instructions, drifts from the privileged start position into the neglected middle as new turns pile on top. An agent loop that appends every tool result pushes its original task description deeper into the middle with each step. In both cases the content that most needs the model's attention is exactly the content that time moves into the low-attention zone, which is why active management of the window, rather than passive accumulation, is necessary to keep important content where the model will actually use it.

How to Place Content to Avoid It

The practical response is to treat the edges of the window as premium real estate and place the highest-priority content there. The system instructions belong at the very start, pinned so that accumulating content never displaces them, and where the model provides a dedicated system role, using it keeps the instructions in the position the model weights most. The single most important piece of retrieved or task content is a candidate for the end of the window, immediately before generation, where recency gives it the strongest pull on the answer. The full set of placement decisions is covered in how to structure the context window, but the core rule is simple: put what matters most at the boundaries and let lower-priority material fill the middle.

The more powerful fix, though, is to avoid a large middle in the first place. Lost in the middle is a problem of long windows, so the same lean-window discipline that fights distraction and rot also shrinks the zone where content gets lost. If aggressive selection keeps the window to the few thousand tokens that actually matter, there is little middle for anything to disappear into, and placement becomes a fine adjustment rather than a rescue. This is why retrieving only the right context and compressing what stays are the first-line defenses: they reduce the total length, and a short window has no dangerous middle. Placement is the tactic you use for the content that remains after you have already made the window as lean as the task allows.

A third technique helps when a window genuinely must be long, such as when a single large document has to be reasoned over in full. Ordering the content by relevance, so the most pertinent passages sit at the top and bottom and the least pertinent fill the center, aligns the model's attention with your priorities. Some systems also re-state the key question or the critical fact at the end of a long context, right before generation, so that even if the original mention got buried, a reminder sits in the high-attention recency zone. These are workarounds for the cases where you cannot make the window short, and they are second best to simply keeping it lean, but they meaningfully reduce the loss when length is unavoidable.

The Role of Memory

A memory layer is one of the cleanest ways to keep windows short enough that lost in the middle stops mattering. Instead of carrying a long history verbatim, where important early facts drift into the middle as the conversation grows, a memory system stores those facts outside the window and selects only the relevant ones back in for the current request. The window then holds a small, current set of facts rather than a long transcript, which keeps the total length low and puts the model's attention on content that all matters. Adaptive Recall does this by ranking stored facts and returning the few that bear on the request, so the window stays lean by construction and the middle never fills with stale history.

This reframes lost in the middle as a symptom of carrying too much in the window rather than a fixed limitation you must design around. The systems that suffer from it most are the ones that accumulate context passively, appending history and retrieval results until the window is long and its center is crowded. The systems that suffer least are the ones that treat the window as a small, deliberately assembled space, refreshed each request from external storage. Managing memory well, covered across the AI memory pillar, is therefore not a separate concern from lost in the middle, it is one of the direct ways to make sure the effect never gets a long middle to work on.

How to Test for It in Your Own System

You do not have to take the effect on faith, you can measure it in your own pipeline with a simple test that mirrors the original research. Take a request whose answer depends on one specific fact, then run it repeatedly while moving that fact to different positions in the window, from the very start, through the middle, to the very end, and pad the rest of the window to a realistic length with relevant but non-decisive content. Record whether the model uses the fact correctly at each position. If accuracy dips when the fact sits in the middle and recovers at the edges, you have reproduced lost in the middle for your model and your window length, and you now know how large the effect is for your specific setup rather than in the abstract.

This test is worth running because the size of the effect varies with the model, the window length, and the kind of content, so the right response depends on numbers you can only get by measuring. A model that shows a small dip at your typical window length may need only light attention to placement, while one that shows a large dip tells you to be strict about keeping windows short and putting decisive content at the edges. Running the test again whenever you change models or materially change your typical window length keeps the decision current, because a model swap can move the curve. Folding this check into the broader practice of measuring context quality turns lost in the middle from a warning you have read about into a measured property of your system that you manage deliberately.