Core concepts stage summary

The agent loop has a precise internal structure, and the most consequential fact about it is negative. At the start of every iteration the model reads its full context: the system prompt, all prior conversation messages, any tool results returned since the last turn, and anything retrieved from memory and injected. It then reasons, and then it either emits a structured request to call a tool or produces a final response. What it does not do is carry anything forward privately. There is no hidden register that survives from one iteration to the next. Every observation is a fresh reading of a growing context window, and the model's apparent continuity across a long task is an illusion produced entirely by the fact that the earlier turns are still in the window being re-read.

That has a blunt engineering consequence. If a piece of information is not in the context window on this iteration, the agent cannot act on it, no matter how recently it was true, how obvious it seems, or how clearly it was stated three turns ago before a truncation strategy removed it. Every design question about agent memory, history management and retrieval is downstream of this one property. It also explains why the boundary between the model and your application is so sharp: the model requests a tool call by producing structured output, and your application layer is what executes the call and injects the result back into the window for the next observe phase. The model never reaches out and touches anything.

The cycle continues until the task is complete or a safety limit is reached, and the second clause of that sentence is not decoration. The stage opens on the pattern the first generation of publicly available autonomous agent frameworks made famous: an agent given an open-ended goal, searching, finding conflicting recommendations, searching again for a comparison, finding more conflict, and repeating that cycle until its token budget ran out. The agent had no step limit, no way to detect that recent steps had produced no new information, and no plan structure that could separate needing more information from going in circles. Termination is a design decision. A hard step counter bounds execution, and a stagnation check that compares recent observations against earlier ones catches circularity before the budget does.

Chain-of-thought prompting elicits step-by-step reasoning from a model before it commits to an answer. The published result is specific and worth holding precisely: generating a series of intermediate reasoning steps significantly improves a large model's ability to perform complex reasoning, measured across arithmetic, commonsense and symbolic reasoning tasks, with the effect emerging in sufficiently large models. The mechanism is not mystical. Reasoning tokens written into the context change the probability distribution over the tokens that follow, so an action chosen after a written plan is drawn from a better-conditioned distribution than one chosen cold. For agents this is usually embedded in the system prompt as an instruction to state what is currently known, what is still needed, and which tool comes next, which produces a second benefit at no extra cost: a written audit trail for the moment something goes wrong.

Three planning strategies cover most agent work, and each is a different answer to the question of when the plan is formed. ReAct interleaves reasoning and action on every step, so the agent writes a trace, calls a tool, reads the result, writes another trace and continues. That is the right shape when the plan genuinely cannot be formed upfront because each step reveals what the next step needs, as in research and exploratory work, and its cost is verbosity that accumulates into real spend. Plan-and-Execute forms the whole plan first and then works through it, which allows a capable model to plan while a faster and cheaper one executes, and which fails when an early step returns something unexpected and the executor has no mandate to revise the plan. Reflection generates an answer, reviews it against the original goal, and revises before returning, which suits quality-critical writing and analysis at the price of roughly doubled latency and cost.

The trap sitting underneath all three is treating reasoning volume as a proxy for reasoning quality. An agent can produce a long, fluent and entirely plausible chain that arrives somewhere wrong, and reflection is vulnerable to the same failure from the other direction, because the model applying the critique is applying the same reasoning patterns that produced the error and frequently cannot see it. This is most acute for factual errors dressed in confident prose. On well-defined tasks a bounded Plan-and-Execute run often beats open-ended ReAct precisely because it forces commitment to a plan instead of accumulating indeterminate reasoning. Measure task completion rate and accuracy. Reasoning length is an input, not a result.

A tool schema is a structured object with three parts: a name, a natural-language description that guides when the tool should be used, and an input schema defining the parameters it accepts. Providers differ only in the outer wrapper, and provider-agnostic frameworks translate between the shapes, so understanding one format is enough to work in both. The input schema follows JSON Schema, whose current version is identified by date rather than by draft number, so a schema that names its version is easier to reason about later than one that does not. The part that decides whether an agent behaves is none of that structure. It is the description, because that is the text the model reads at every reasoning step in order to choose.

An effective description answers three questions: what the tool returns, when the model should call it, and when the model should explicitly not call it. The third is the one teams omit, and its absence is why agents call tools that add latency and cost for no benefit. That gives four tool selection failure modes with clean diagnoses. The wrong tool is chosen when two descriptions overlap or the correct one is less specific than its neighbour. The right tool is called with wrong parameters when the parameter descriptions are incomplete. No tool is called at all when the description does not match the vocabulary users actually type, which is fixed with synonyms and example triggers. And a tool is called unnecessarily when nothing in the description tells the model when to stay away.

Tool count matters for the same reason. The model must read every description and discriminate between them under a fixed attention budget, so selection accuracy degrades as the set grows, and the module gives a working rule of thumb in the region of fifteen to twenty tools before confusion sets in. The discipline is to start from the minimum set that can accomplish the task, add tools one at a time, and measure whether each addition raises or lowers task completion rate before keeping it. This is also the structural answer to the failure the module opens on, where a customer-facing assistant answered a policy question from its training data and the organisation was held responsible for what it said. A description that states plainly that policy questions are always answered from this tool and never from memory routes the question to a live, authoritative source. No amount of temperature tuning or model upgrading does that.

The model never calls your function. It determines when a tool is appropriate from the request and the tool's description, and returns a structured call naming the tool and its arguments. Your application code interprets that, decides whether the call is permitted, validates the arguments, executes the actual function, and sends the outcome back as a tool result for the model to read on its next observe phase. That separation is the single most useful architectural fact in the module, because it locates every control you care about. Authorisation, rate limiting, argument validation, redaction and audit logging are all application concerns. Attempting to hold them in a system prompt is asking a probabilistic component to enforce a policy, which it will do most of the time.

Validation belongs at that boundary in two layers, because the model generates arguments from a probability distribution rather than from logic. It will produce a date in an unexpected format, a number outside a valid range, or a well-formed string that references a resource which does not exist. A schema constraint catches structural mismatches, and a runtime validation layer catches the semantic errors a schema cannot express, such as a recipient field that contains a person's name rather than an address. When validation fails, serialise the failure and return it as a tool result naming the field, the constraint and the value supplied, so the model has enough to correct itself. For tools that write data or trigger anything irreversible, pair validation with an authorisation check that the current session is permitted to invoke this tool with these arguments, before execution rather than after.

Failure handling follows the same principle, and the rule is that errors are results, not exceptions. An unhandled exception ends the agent loop and the model never learns anything happened. A structured error injected as a tool result keeps the loop alive, and the model can read what went wrong, retry with corrected arguments, fall back to a different tool, or tell the user the action failed. The four failure classes each want a different response: invalid parameters should return a structured error and let the model recover, transient external failures deserve bounded retry with backoff, authentication failures should fail immediately because retrying an expired credential only wastes time, and unexpected output such as a document where structured data was expected should be caught at the parse step and reported as an error rather than allowed to propagate. Sanitise what goes back to the model. Log the full trace server-side and return a description of what failed without exposing internals.

The context window is working memory: immediately available with no retrieval step, finite, charged per token, and gone at the end of the session. That is enough for a short task and useless for an assistant that has to recall a customer's issue from last week or a researcher that would otherwise re-read the same corpus on every query. So agent memory divides three ways. In-context memory is whatever is in the window right now. External memory is structured storage outside the model, such as a relational database or a key-value store, reached through a tool call, and it is the right choice when you know exactly what you are looking for: customer records, session history, configuration, anything queried predictably by key. Semantic memory is a vector store that searches by meaning rather than by exact match, and it is the right choice when the relevant content cannot be predicted from the wording of the query.

Choosing wrongly here is one of the most common architectural errors in agent design, and it goes wrong in both directions. Loading a corpus of tens of thousands of documents into a context window is not a tuning problem, it is impossible. Standing up a vector store to look up a customer by identifier is slower, more expensive and less exact than the database query it replaced. The question to ask is not how much data there is but how the data will be reached: by key, by meaning, or not at all because it is already in front of the model.

Semantic memory works because an embedding is a numerical vector representation of text in which similar meanings sit close together, so a query and a passage that share no words at all can still be near neighbours. A question about cancelling a subscription and a passage headed with account termination language have no vocabulary in common, and keyword search returns nothing while vector search ranks the passage highly. Embedding models differ in vector dimension, in cost and in domain fit, with some tuned for code and technical content and others for multilingual corpora, and matching the model to the domain of the corpus measurably improves retrieval. The module names the current options; treat those names as perishable and the selection criterion as durable.

A retrieval-augmented pipeline has two phases that run on different schedules. Ingestion runs once, and again whenever the corpus changes: documents are split into chunks of a few hundred tokens with deliberate overlap at the boundaries, each chunk is converted to a vector by an embedding model, and the vectors and their text are written to a vector store. Retrieval runs on every query: the query is embedded with the same model used at ingestion, the store performs an approximate nearest-neighbour search, and the closest chunks are injected into the context window as supporting material for the model to answer from. Using a different embedding model at query time than at ingestion time is a silent, total failure, because the two sets of vectors do not share a space.

The dominant failure point is chunking, not search. If a policy paragraph is split mid-sentence, neither resulting chunk contains the complete answer, and no amount of retrieval quality recovers information that was destroyed during ingestion. Overlap between adjacent chunks mitigates the boundary problem, and splitting at natural semantic boundaries rather than at a fixed token count does better still at higher implementation cost. The top-k setting is the other lever and it is a straight precision against recall trade: retrieving a single chunk maximises precision and loses badly whenever the best chunk is not ranked first, while retrieving many raises the chance the right material is present at the cost of filling the window with content the model then has to reason past. It is also worth stating plainly that retrieval is not automatically better than the model's own knowledge. An ambiguous query retrieves the wrong chunks and produces an answer confidently grounded in the wrong thing, which is why retrieval quality and generation quality have to be evaluated separately.

Long conversations exhaust the window, and three strategies manage that with different losses. Full history appends everything: simplest, and it eventually hits the limit, so it suits short task-focused exchanges. A sliding window keeps the most recent messages plus the system prompt: cheap and fast, and it discards early context, so it suits conversations where nothing said early on constrains what happens later. Summarisation compresses older messages into a summary and keeps recent messages verbatim, which preserves more than a sliding window and still loses things, characteristically the constraint stated once at the very beginning that the summariser judged incidental. Keep the most recent messages verbatim, summarise only older stretches, and never summarise the system prompt. All three inherit a governance obligation as soon as they persist across sessions, because conversation history containing user data falls under data protection law: tell people, set a retention period, and be able to delete on request.

The instinct when building an agent is one agent holding every tool. That works for focused tasks and fails predictably as tasks grow: the window fills with tool history irrelevant to the current step, the model confuses tools intended for different sub-tasks, and a failure anywhere blocks everything. Six patterns give you the vocabulary to see those failures coming. A single agent is one model, a set of tools and a loop, and it is the most underused of the six; it suits a well-defined task in one domain with a small tool set. A router puts a lightweight classifier in front and delegates to a specialised agent, which keeps each agent's tool set clean and lets a cheap model make the routing decision; if the routing taxonomy has grown to ten categories, the problem has probably been framed wrongly. A supervisor orchestrates sub-agents, assigns work and synthesises the results, which buys parallelism at the cost of a real hazard: a supervisor synthesises what its workers report, so a worker that invents something can have it laundered into the final answer unless validation lives in the workers' own tool calls.

The remaining three are about the shape of the work rather than the shape of the team. Map-reduce applies an operation across many similar items, potentially concurrently, then reduces the results, which makes it the right and cheapest answer for batch classification and extraction over large collections, particularly with a fast model in the map phase and a capable one only in the reduce. A chain passes the output of each step as the input to the next, which localises failures to a named step, makes each step independently testable, and allows quality gates between stages. Reflection has the agent critique and revise its own output before returning it, which improves quality when the self-critique is reliable and does nothing when the same reasoning that produced the error is applied to finding it. The research on this pattern is worth reading precisely because it makes the mechanism explicit: the agent is reinforced not by updating weights but through linguistic feedback, reflecting verbally on task feedback and keeping that reflective text in an episodic memory buffer so later attempts are informed by earlier ones. A reflection step that remembers its own previous critiques is a different and stronger thing than a single-pass self-review.

The selection rule is the same every time. Start with the simplest pattern that satisfies the requirement and move up only when there is a specific, measurable problem: context overflow, tool confusion, a genuine need for parallelism, or a quality improvement that pays for the latency it costs. Multi-agent architectures are not automatically more capable. They add orchestration complexity, extra model calls, extra cost, and the risk of errors compounding across agent boundaries, and a single agent with a focused tool set and a well-written system prompt frequently beats them on the same task. If the routing logic can be written as a decision tree, it very probably does not need a model at all.

Framework choice should follow pattern choice, not lead it. Graph-based orchestration frameworks model a workflow as nodes joined by edges over a shared, typed state object: each node receives the current state, does its work and returns a partial update that the framework merges before the next node runs, and conditional edges route to different nodes based on a function that reads that state. That is how branching is expressed, by checking whether the last message contained a tool call, whether a step counter has passed its limit, or whether a flag was set upstream. Persistence is the other half: saving state at node transitions lets a long-running workflow resume from where it left off after an interruption, and it is what makes a human approval gate possible, where execution pauses at a defined node, the current state is surfaced for a person to inspect and modify, and the workflow resumes only once they have decided. For any agent that sends messages, moves money, deletes records or changes production data, that gate is an architectural requirement rather than a feature. Role-based frameworks suit work that genuinely maps onto a team of specialists, and a thin client library over the API remains the right choice when a project wants direct control of the loop and auditability more than it wants abstraction.

The distinction the module builds the whole architecture on is between agent state and conversation history, and conflating them is what produced the memory management failures it opens with, where agents that worked in development degraded in production once conversations ran long: history was silently overflowing, being truncated in ways that removed material the task depended on, or being assembled into message lists the model then misread. Conversation history is the list of messages the model reads in the context window, and its job is to give the model what it needs to decide the next move. Agent state is everything the orchestration code tracks in order to decide which node runs next: step counters, retry limits, error counts, progress flags, extracted data, intermediate results. A document processing agent tracks pages processed and extraction errors, and none of that belongs in the window, because the model only needs the current page and the instruction. Keeping orchestration state out of the context saves tokens, relieves window pressure, and stops the model reasoning about metadata that was never addressed to it. Manage the two with separate, independently testable components, and choose framework complexity proportional to the problem; a simple assistant with a handful of tools and no branching gets nothing from a graph abstraction except a graph abstraction.

Concurrency is the last piece and it is governed by dependencies rather than by preference. Executing tool calls one at a time costs the sum of their latencies; executing independent ones together costs roughly the slowest of them. The standard Python primitives for this are the async and await syntax and the library built around them, which exists to write concurrent code and to run coroutines concurrently, and which is a particularly good fit for the input and output bound work that tool calls almost always are. Graph frameworks support async nodes natively. What cannot be parallelised is anything with a real dependency: if a step needs an earlier step's output to determine its own arguments, it waits, and forcing concurrency onto dependent operations produces races and wrong answers rather than speed. Work out the dependency graph first, then parallelise what the graph says is independent. Several lookups against different sources for the same question are almost always independent and almost always worth running together.