Agent Loops: The Run, Observe, Decide Cycle

The Three Phases

Run: The model receives the current context, which includes the system prompt, the original task, the conversation history (all previous reasoning, tool calls, and tool results), and any memory retrieved from prior sessions. The model processes this context and generates a response that contains either tool calls (actions the agent wants to take), text output (a partial or final response), or both. This phase is where the model's reasoning happens: it integrates everything it knows about the task, evaluates what it has learned from previous iterations, and decides the next action.

Observe: The system executes any tool calls the model generated. This might involve calling an API, querying a database, running code, reading a file, or performing any other action the agent's tools support. Each tool execution produces a result, which could be data (search results, database rows, file contents), a status (success, failure, error message), or a side effect confirmation (email sent, record updated). These results are appended to the conversation history so the model can see them in the next iteration.

Decide: The system determines whether another iteration is needed. The most common decision rule is: if the model's response contained tool calls, execute them and loop back to the run phase. If the model's response contained only text (no tool calls), treat it as the final answer and return it to the user. This simple rule works because the model itself signals when it is done, by choosing to respond with text rather than calling another tool. Additional termination conditions include the iteration limit (hard stop after N iterations), cost limit (stop when token budget is exhausted), and timeout (stop after T seconds of wall-clock time).

Context Growth Across Iterations

The central engineering challenge of the agent loop is that the context window grows with every iteration. Each iteration adds the model's reasoning text, the tool call specification, and the tool result to the conversation history. After 10 iterations, the history might contain 20,000 to 50,000 tokens, depending on the verbosity of the model's reasoning and the size of the tool results. This growth creates three problems: cost (every token is paid for on every subsequent model call), latency (larger contexts take longer to process), and quality degradation (the model's attention is diluted across more tokens, making it harder to focus on what matters).

The first mitigation is to truncate tool results. Most tool results contain far more information than the agent actually needs. A web search might return 10 results with full snippets, but the agent only uses 2 or 3. A database query might return 100 rows when the agent only needs aggregate statistics. Truncating tool results to a reasonable length (1,000 to 3,000 tokens per result) before adding them to the history significantly reduces context growth without losing the information the agent actually uses.

The second mitigation is to summarize completed work. After the agent has used a tool result and moved on to the next step, the full tool result is no longer needed. Replacing it with a short summary ("Web search found that Acme Corp reported $4.2B revenue in Q1 2026") preserves the key finding while freeing context space. This can be done by the model itself (asking it to summarize each tool result after using it) or by a separate summarization step between iterations. The context compression guide covers these techniques in detail.

The third mitigation is to offload to memory. Instead of keeping all intermediate results in the context window, write them to a memory layer and retrieve them only when they are needed again. An agent that has gathered data from five sources can store the findings in memory, clear them from the context, and recall the specific findings relevant to its current reasoning step. Adaptive Recall provides this capability through its store and recall tools, letting agents maintain a clean context window while retaining access to everything they have learned. The working vs long-term memory guide covers the architecture.

Failure Modes

Stuck loops: The agent repeats the same action without making progress. This happens when a tool returns an unhelpful result and the agent does not know how to try a different approach, or when the agent misinterprets a tool result and keeps trying the same failed approach. Detection: track the last N tool calls and flag when the same tool is called with identical or near-identical parameters. Mitigation: inject a message into the context telling the agent it appears to be stuck and should try a different approach.

Goal drift: The agent starts working on the original task but gradually shifts to a tangentially related topic as intermediate results pull its attention. This is especially common in research tasks where each search result opens new avenues to explore. Detection: periodically compare the agent's current activity to the original task description. Mitigation: include the original task prominently in the system prompt and periodically reinject it into the context with a reminder: "Your original task is [X]. Stay focused on completing it."

Context overflow: The conversation history exceeds the model's context window limit, causing truncation of either the system instructions or early conversation turns. When the system instructions are truncated, the agent loses its behavioral constraints and may act unpredictably. When early turns are truncated, the agent may lose the original task description. Detection: track the total token count before each model call and compare it to the model's window limit. Mitigation: apply the context management strategies above (truncation, summarization, memory offloading) proactively, before the window is full.

Cascading errors: An error in an early iteration, such as a wrong search query or a misinterpreted tool result, leads the agent down a wrong path, and subsequent iterations build on the wrong foundation. By the time the error becomes apparent, the agent has invested most of its iteration budget in the wrong direction. Detection: this is the hardest failure mode to detect automatically because the agent's reasoning appears internally consistent. Mitigation: for critical tasks, add periodic self-checks where the agent reviews its progress against the original objective, or run a separate validation agent that evaluates the primary agent's intermediate results.

Designing the Termination Condition

The termination condition determines when the loop stops, and getting it wrong in either direction causes problems. If the loop terminates too early, the agent returns incomplete or low-quality results. If it terminates too late, the agent wastes tokens and money on iterations that add no value. Most agents use a combination of termination signals rather than relying on a single one.

The primary signal is the model's own decision. When the model generates a text response without any tool calls, this signals that it believes the task is complete. This is the cleanest termination because the model has the best understanding of whether the task objective has been met. However, models sometimes terminate prematurely (generating a partial answer rather than gathering more information) or refuse to terminate (continuing to call tools when it already has enough information). Tuning the system prompt to be explicit about when the task is "done enough" helps: "When you have gathered sufficient information to answer the question confidently, provide your final answer. Do not continue searching once you have found a clear, verified answer."

The secondary signals are hard limits that prevent runaway loops. A maximum iteration count (typically 10 to 25 for most agents) catches cases where the model never signals completion. A cost ceiling (maximum total tokens per task) catches cases where large context windows make individual iterations expensive. A wall-clock timeout catches cases where tools are slow or unresponsive. When any hard limit is hit, the agent should be prompted to produce its best answer with the information gathered so far, rather than simply crashing. This is the graceful degradation pattern covered in the production reliability guide.

For multi-step tasks with clearly separable phases, consider phase-based termination where each phase has its own iteration budget. A research agent might have 5 iterations for searching, 3 for reading and extracting, and 2 for synthesis. This prevents the agent from spending its entire budget on search and having nothing left for synthesis, which is a common failure pattern when all phases share a single global budget.