How to Manage Conversation History
Left unmanaged, history is the source that breaks a chatbot first. Every turn adds tokens, and a conversation included verbatim will eventually overflow the window or, well before that, push the system instructions into the neglected middle and drown the current request in old chatter. The naive fix, a sliding window that keeps only the last few turns, throws away facts the user stated earlier that still matter. These steps keep history bounded while preserving what the conversation established, using summarization for the recent past and a memory layer for the durable facts.
Step 1: Set a history budget
Decide up front how many tokens of the window conversation history is allowed to occupy, as part of the overall context budget. This allocation is what forces the management to happen: without it, history grows until something breaks, and with it, history has a ceiling it must live within on every turn. The budget should leave ample room for the system instructions, the current request, and whatever retrieved knowledge or memory the request needs, so history gets a bounded slice rather than the run of the window. Setting this number is the decision that turns history management from an afterthought into a rule the pipeline enforces every request.
Step 2: Keep the most recent turns verbatim
Preserve the last several exchanges word for word, because the immediate thread of a conversation lives in its most recent turns and summarizing them loses the nuance the next response depends on. When a user asks a follow-up like fix that or what about the second one, the referent is in the recent turns, and only the verbatim text resolves it. Reserve a portion of the history budget for this recent window and keep it literal. How many turns to keep depends on how long individual turns run, the goal is to retain enough that the current exchange is fully grounded, while everything older than that band becomes a candidate for summarization.
Step 3: Summarize older turns
Roll the turns older than the verbatim band into a running summary that preserves what still matters: the decisions made, the facts established, the open questions, and the user's stated goals, while dropping the conversational filler. Maintain the summary incrementally, updating it as turns age out of the verbatim band rather than regenerating it from scratch each time, which keeps the cost bounded and the recap stable. The summary is a compression of history, so the general cautions in how to compress context apply, above all the protected-facts discipline: a summary that drops the one detail the user will reference later has failed, so the prompt that produces it should be told what kinds of facts must survive.
Step 4: Extract durable facts to memory
Some facts a user states should outlive the conversation entirely, their name, their preferences, their configuration, the constraints of their project, and these belong in a long-term memory layer outside the window rather than in a summary that exists only for this session. As the conversation proceeds, extract these durable facts and write them to memory, so they can be recalled in future sessions and do not have to be rebuilt every time the user returns. This is the write strategy of context engineering applied to dialogue, and it is what separates an assistant that remembers a user across days from one that resets every session. Adaptive Recall is built for this step, storing extracted facts with confidence scores and returning the relevant ones on later requests, so the memory the conversation produces becomes available without bloating the current window.
Step 5: Reconcile contradictions
Users change their minds, and a naive history keeps every position they ever took, which creates the context confusion where the model cannot tell the current preference from an abandoned one. As you summarize and as you write facts to memory, reconcile contradictions so the current state is authoritative: when a user switches from option A to option B, the summary and the stored fact should reflect B and demote or drop A, rather than carrying both as equal. The same applies to corrections, when a user fixes a detail they gave earlier, the corrected version supersedes the original. Keeping history and memory reconciled is what ensures the window carries a single coherent view of what the user wants rather than a contradictory pile of everything they have ever said.
Step 6: Drop the rest under budget
Once the recent turns are preserved, the older ones summarized, and the durable facts written to memory, the original verbatim text of the old turns has served its purpose and can be discarded. This is the step that actually keeps history within budget, because everything of value has been captured in a more compact form, the summary and the stored facts, so the bulky original transcript no longer needs to occupy the window. Dropping it is safe precisely because the preceding steps extracted what mattered first. The result is a history section that stays within its allocation no matter how long the conversation runs, holding a compact summary plus the live recent turns, while the durable knowledge lives in memory ready to be selected back when relevant.
Manage history with a budget, verbatim recent turns, a running summary of older turns, durable facts written to a memory layer, reconciliation so the current state wins, and disposal of the raw old transcript once its value is captured. This lets a conversation run indefinitely while the window stays lean.
Summary Plus Memory Beats Either Alone
The two mechanisms in this method, the running summary and the memory layer, handle different halves of the problem and are weakest alone. A summary alone captures the arc of the current session but is discarded when the session ends, so the assistant forgets the user by their next visit. A memory layer alone captures durable facts across sessions but is not the right tool for the fine-grained thread of the live conversation, the exact phrasing of the last three turns that a follow-up depends on. Using both, the summary for within-session continuity and memory for across-session persistence, covers the full range, from the immediate follow-up to the fact a user mentioned last month. The distinction between session history and persistent memory is developed further in whether memory is part of context engineering.
History Management for Agents
Agents face the same problem in a sharper form, because an agent loop appends a tool call and its result on every iteration, so its history grows faster than a human conversation and its window fills within a single task. The same method applies with one shift in emphasis: agents summarize aggressively, rolling completed sub-tasks into a short result the moment they finish so the budget stays available for the remaining work, and they lean heavily on writing intermediate state to a scratchpad or memory rather than carrying it in the window. The agent-specific patterns are covered in context engineering for AI agents, but the core discipline is identical, keep the recent steps live, compress the older ones, persist the durable results, and drop the raw trace once its value is captured.
Choosing How Many Turns to Keep Verbatim
The one tuning decision this method leaves open is the size of the verbatim band, how many recent turns to keep word for word before summarization takes over, and getting it right is what makes the difference between an assistant that resolves follow-ups smoothly and one that loses the thread. Keep too few turns and the assistant forgets the immediate context, so a user who says apply that to the other file gets a confused response because the turn that defined that already got summarized. Keep too many and the verbatim band eats the budget that retrieval and memory need, and long turns can push the window into the length where quality degrades. The band should be large enough that any reasonable follow-up finds its referent in the literal recent turns, and no larger.
Because turns vary in length, the better way to size the band is by tokens rather than by a fixed turn count. A conversation of short exchanges can keep many turns verbatim within a modest token allowance, while a conversation with long, detailed turns fills the same allowance in just a few, so measuring the band in tokens keeps it consistent across both. Set the band as a token slice of the overall history budget, keep the most recent turns that fit within it verbatim, and let everything older roll into the summary. When you observe follow-ups failing because their referent was summarized too early, widen the band, and when the verbatim text is crowding out retrieval, narrow it. This is one more application of measuring rather than guessing, and folding it into context quality measurement lets you tune the band on evidence instead of intuition.