Advanced mastery stage summary

Prompt engineering is fast, cheap and reversible, which is why it is the right starting point for almost every task. Fine-tuning is slower, more expensive and much harder to undo. The module is blunt about the test that separates them: fine-tuning answers the problems prompting cannot solve, and the load-bearing word is cannot. If a well-built prompt reliably produces the output you need, a fine-tune adds cost and operational complexity and buys nothing. The case for training only opens once you have tried carefully and the model still fails on your specific distribution of inputs.

Four situations make that case. You need a consistent output format that currently costs several hundred tokens of instruction on every single call. You need domain conventions the base model does not have. Your request volume is high enough that inference cost dominates, in which case a smaller specialised model can do the same narrow job for a fraction of the per-call cost. Or your data cannot leave your infrastructure, which rules out an external interface regardless of quality. The module's indicators for choosing to train are high volume, consistent failure under careful prompting, an exact and repeatable output format, a latency requirement, or a privacy constraint that forbids an external service.

Frame the comparison as a sum rather than a preference. On one side sits the one-time cost of collecting and reviewing data, running the training, and building the evaluation that tells you whether it worked. On the other sits the recurring cost of longer prompts and a more capable model, multiplied by your request rate for as long as the system runs. Neither number is knowable without the volume, so a fine-tuning proposal that does not state the volume is not yet an argument. And note the asymmetry: the recurring side keeps accruing, while the one-time side has to be paid again every time the task definition moves.

One confusion is worth killing here because it drives bad projects. Fine-tuning is not a cure for wrong answers. It adjusts style, format and behaviour, and it does not install facts into the weights in any retrievable way, so a fine-tuned model will produce confident, correctly formatted, entirely fabricated output just as readily as the base model did. When the failure is factual rather than stylistic, retrieval is the tool: give the model the evidence at inference time instead of hoping it was absorbed during training. The two combine well, and the module recommends exactly that pairing for tasks where accuracy of content matters.

Full fine-tuning updates every weight. The memory that requires is not just the weights themselves: you also hold activations, gradients and optimiser state, which is how a model that sounds affordable at rest turns into a multi-card requirement during training. Parameter-efficient fine-tuning takes the other route and leaves the overwhelming majority of the weights frozen, updating only a small set of additional parameters. Low-rank adaptation is the technique that made this the default. Rather than modifying a weight matrix, it inserts two small trainable matrices beside it, so the effective weight during the forward pass is the original plus their product, and only those two small matrices receive gradients.

The reason this works at all is an empirical claim worth stating explicitly, because it is what you are betting on: the change that fine-tuning makes to a weight matrix has intrinsically low rank, so it can be represented accurately by a product of two thin matrices without losing what matters. That is why a run can report a trainable fraction well under one per cent of the model and still change the model's behaviour on your task. It also explains the main knob. The rank sets how much capacity the correction has, so a higher rank means more trainable parameters, more memory and slower training, with quality gains that flatten out. The module treats a rank in single or low double digits as typical, and pairs it with a choice of which projections inside the attention block to attach adapters to.

Quantised low-rank adaptation adds one more step and it is the step that changed who can do this work. Before training begins, the frozen base weights are compressed to four bits, which cuts the resident memory of the part you are not training. The trainable adapters stay at higher precision, so the gradients that actually matter are not being computed through a coarse representation, and training quality holds up. The published result behind the technique is that this brings a large model's fine-tuning inside a single high-memory card while preserving the task performance of full sixteen-bit fine-tuning, using a four-bit data type designed for normally distributed weights, a second round of quantisation applied to the quantisation constants themselves, and paged optimiser state to absorb memory spikes.

Two practical consequences follow. First, the artefact you produce is small: a set of adapter matrices rather than a whole model, which makes it cheap to store many task-specific adaptations against one base. Second, the hardware question and the quality question have come apart. Whether you can run the training is now mostly a memory arithmetic problem you can answer in advance, which means the risk in a fine-tuning project has moved almost entirely into the data and the evaluation.

The model can only learn what is in the data, and it will learn the mistakes with the same fidelity as the patterns. The module's guidance is that a few hundred diverse, carefully checked examples consistently beat several thousand poor ones, which inverts the instinct to collect as much as possible before starting. The standard instruction format pairs each example with an instruction, an optional input and the desired output, and what the model takes from it is the pattern of following instructions rather than the specific content of any one of them.

Run an audit before you run the training. Every instruction should be specific and consistent in shape, because inconsistency in the prompt side teaches the model that the format does not matter. Outputs should be correct and ideally reviewed by someone who knows the domain. No example should carry personal data. Edge cases and known failure modes should be present deliberately rather than filtered out for being awkward, since a set of clean easy examples produces a model that is confident and useless at the boundary. And split the data into training, validation and test portions before anything else touches it, so the set you eventually judge on was never trained on.

Generating training data with a more capable model is a legitimate and often effective way to get to volume quickly, and the trap is treating the output as finished. Generated examples contain errors, and a fine-tuned model learns those errors confidently. Sample the generated set, have a person read it, and fix or discard what is wrong. Automatic generation accelerates collection; it does not replace quality control.

Evaluation is where fine-tuning projects most often flatter themselves. A falling training loss says the optimiser is working, not that the model is better at your task, so measure on the held-out set with metrics that match the job: precision, recall and their harmonic mean for extraction, overlap against reference outputs plus human judgement for generation, and schema validity together with field-level accuracy where the output is structured. Compare against two baselines, not one: the base model, and the base model with your best prompt. If the fine-tune does not beat the prompt, you have bought a maintenance burden. When accuracy plateaus, resist the urge to double the dataset. At a high score the general pattern is already learned and what remains is a set of specific failure modes, so read the failing cases, work out what they have in common, and add examples that target that.

What separates a prototype agent from an enterprise one is rarely the model. It is queue depth, tail latency under load, cost per query at volume, the completeness of the audit trail, version control across environments, and the organisational reality of several teams sharing one piece of infrastructure. None of those exist on a laptop, and all of them arrive at once when the system becomes something people depend on.

The central architectural move is to stop making the caller wait. In a synchronous design every request occupies a connection for the whole duration of the work, so a traffic spike raises latency for everyone and a slow upstream model makes every user wait behind it. An asynchronous queue splits submission from processing: the caller submits a task and receives an identifier, a pool of workers pulls from the queue independently, and the result is collected when it is ready. Three components carry it. A gateway authenticates, rate limits and enqueues. Stateless workers hold no memory between tasks, call the model, run tools and write results to storage. A results store makes the outcome retrievable by identifier.

Statelessness is what makes scaling trivial rather than delicate. Because a worker carries nothing between tasks, adding capacity is adding processes, and removing it is stopping them. On a container orchestrator this becomes automatic: the autoscaler adjusts the number of running instances to match demand, and it can be driven by a custom or external metric such as queue depth rather than only by processor utilisation, which matters here because an agent worker spends most of its time waiting on a network call and looks idle by processor measures while being entirely saturated.

The queue also gives you two properties that are expensive to retrofit. Backpressure: when the upstream model throttles, workers slow, the queue lengthens, and callers are not directly affected because they were never waiting. And redelivery: if a worker dies mid-task the message was never acknowledged, so another worker picks it up. Getting either of those into a synchronous design means scattering retry logic through every piece of calling code.

One user request can produce several model calls, several tool executions and a handful of database queries across different services. When it goes wrong you need the sequence, the timing and the inputs and outputs of each step, and unstructured logs will not give you that at scale because they do not record what caused what. Distributed tracing does. You wrap each logical unit of work in a span: the run as a whole, each model call, each tool execution. Every span in one request shares a trace identifier, and each child span records the identifier of its parent, so the collection reassembles into the path the request actually took. Look up a failing task and you see every step, its duration and its output in one view. Logs still matter for detail; carry the trace identifier into them so the two line up.

Alongside traces, keep a small set of metrics you would actually act on. The module's four are request count labelled by success and failure, response latency as a histogram so you can read percentiles rather than an average that hides the tail, tokens consumed per model, and current queue depth. Alert on an error rate above a stated fraction and on the ninety-ninth percentile latency crossing your service commitment. A metric with no threshold and no owner is decoration.

Cost at enterprise volume is set by routing, not by negotiation. The observation the module builds on is that the difficulty of the work varies enormously while the model handling it usually does not. A short classification does not need the most capable model available; a multi-step reasoning task or code generation does. Sending each task to the least expensive model that handles it reliably is the single largest lever, and it is an architectural decision because it requires a classification step, a routing table and a way to measure whether the cheap path is still good enough.

Pair routing with an enforced budget. Define a token ceiling per agent or per customer, warn well before it is reached, and stop hard when it is. Without a hard stop, one runaway loop or one successful injection can consume a day's spend in minutes, and you will find out from the invoice. Treat the budget as an operational control of the same kind as a rate limit, not as an optimisation to get to later.

Guardrails are programmatic constraints on what goes into an agent and what comes out of it, and their value comes from being independent of the model. This is the distinction to hold on to. An instruction in the system prompt telling the model never to emit payment card numbers lives inside the context window, which is the same channel untrusted input arrives on, so a sufficiently well-crafted injection can talk the model out of it. A check that runs over the response in application code is not in the context window at all and cannot be argued with, whatever the prompt said.

Grade them by severity rather than applying one response to everything. Blocking guardrails stop execution and return an error, which is right for a constraint that must never be violated, such as a recognisable card number in an output or a response that breaks a contract with a downstream consumer. Warning guardrails let the response through but log it and flag it for human review, which suits things that are undesirable rather than forbidden. Logging guardrails record the event for audit and change nothing. Matching the type to the severity is what keeps the blocking set small enough to be trusted.

Regulated deployments add requirements that are architectural rather than procedural. Data residency means certain data must be processed and stored inside a particular jurisdiction, which a self-hosted model or a regional endpoint satisfies and a single central interface does not. Audit completeness means every assisted decision must be reconstructable afterwards: who asked, what was supplied, which model version answered, what came back and when, held immutably for the retention period, which in financial services is measured in years. And decisions that affect a person, such as credit, insurance pricing or hiring, need a human review queue with a documented threshold for what routes to it.

Model version pinning is the control that teams most often discover they needed. Providers update models continuously, and an update can change the output for an identical prompt, which in a regulated context means the system's behaviour changed with no change record. Pin the version, run a regression suite against a labelled test set before moving, and put the upgrade through the same approval as a code release. The module names ISO/IEC 42001 as the management-system framework around this work and the EU AI Act transparency obligations as the legal driver for disclosing what the system is and what it cannot do; read both at source before writing a compliance claim.

A container packages the application with its dependencies so it runs the same way in development, staging and production, which is what makes an automated build and deploy pipeline possible at all. The module's four practices for a production image are worth memorising because each removes a specific class of incident. Use a multi-stage build so the build tooling stays in the build stage and only the artefacts needed at runtime are copied forward, giving a smaller image with less attack surface. Create a non-root user and run as it, since many container escape techniques need root. Copy dependency manifests before application code so the dependency layer stays cached and ordinary rebuilds take seconds. And expose a health endpoint so the orchestrator and the load balancer can tell a live instance from a wedged one and stop routing to it.

For monitoring, the stage adopts the four golden signals from the site reliability engineering literature: latency, traffic, errors and saturation. Latency should be a histogram, because the average is the least interesting number in it: the median tells you what is typical, the ninety-fifth what most people experience, and the ninety-ninth is where the worst requests live, which is the one that generates complaints. Traffic is requests per second against a baseline from the previous week, so a sudden multiple flags a runaway loop or an unexpected caller. Errors is the failing fraction over a window rather than a raw count. Saturation is how close you are to a limit, which for an agent means queue depth and consumed token budget.

Agents need a fifth signal, and the module's opening case is the argument for it. A system accumulated conversation and tool output in the context window; as volume grew, a subset of requests produced long tool-call sequences, the context approached its limit, earlier content was dropped, and the agent began answering without the instructions that told it what to do. Quality degraded quietly and the error rate crept up under the alert threshold for hours. Nobody had instrumented context utilisation, because it is the one resource that grows with every step and produces bad answers long before it produces an exception. Measure it as a percentage of the limit and alert well below the limit.

A split test runs the current configuration and the proposed one at the same time on a share of real traffic and measures which is better before you commit. Two details make it valid. Assignment must be consistent, so the same user always lands on the same variant for the same experiment; hashing the user identifier together with the experiment name gives you that deterministically with no stored state, and inconsistent assignment both confuses users and contaminates the measurement. And the success metrics must be fixed before the experiment starts, not chosen afterwards from whatever moved. For agents the metrics that matter are completion rate against the intended task, tokens spent per successfully completed task, error rate, and a satisfaction signal where you have one. Run until you have enough requests per variant to distinguish a real effect from noise; watching a dashboard for an afternoon is not a result.

Blue-green deployment is the release counterpart. The current version keeps serving while the new one is deployed alongside it with no traffic, then traffic is shifted in stages with a pause to watch at each step. Because the previous version is still running, reversing is a routing change measured in seconds rather than a redeploy measured in minutes. The discipline that makes it work is writing the rollback conditions down before the deployment starts: an error rate above a stated level sustained over a stated window, a tail latency above a stated ceiling, a cost per request that jumps by more than a stated proportion, or any new exception type appearing at more than a trickle. Automate the check so the system reverses itself without needing someone to be awake, and keep the alert threshold and the rollback threshold as two separate numbers, because the first calls a human and the second acts alone.

Latency work has its own discipline: profile before you optimise. The instinct is to attack the model call because it is the most visible and most expensive-sounding step, and the common finding is that application overhead dominates for simple tasks, whether that is serialisation, a synchronous database query or a slow tool interface. Once you know where the time goes, three techniques do most of the work. Stream the response so the user sees output while generation continues, which removes perceived latency without changing generation speed. Cache the stable prefix of a long prompt so repeated calls do not pay to reprocess it. And run independent tool calls concurrently, which collapses several sequential waits into the duration of the slowest one. For conversations that grow, add a sliding window or a summarisation step so the context problem from the opening case cannot recur.

The final module is about capabilities that change how you would design a system, and the principle it gives for using them is the useful part: build for today's reliable capabilities and leave extension points for tomorrow's, rather than betting an architecture on a pre-release feature or building an expensive bespoke mechanism for something that is about to become an interface parameter. The gap between a demonstration and something you can run unattended is measured in the reliability of the full input distribution, the cost at volume, whether the safety boundary can be stated, and whether the same instruction produces a testable result twice.

The first capability is deliberation before answering. A standard generation produces the answer directly; a reasoning mode generates an extended internal working first, exploring approaches, catching its own errors and revising before committing. The module frames this with the familiar dual-process picture of fast automatic thinking against slow deliberate thinking. What matters operationally is that the deliberation is generated at inference time, so it costs tokens and time, and the benefit is real on multi-step mathematics, debugging, architectural trade-offs and scientific reasoning while being close to nil on factual lookup, summarisation, translation and conversation. That asymmetry makes the correct pattern a routing decision: classify the request, spend the deliberation on the minority that need it, and leave the rest on the fast path. The current documentation makes two mechanics durable even as the parameter names move. Internal reasoning is billed as output, so the budget for it has to leave room for the answer or you get deliberation and no result. And the budget is a target rather than a hard cap, with the overall output limit remaining the real ceiling. Check the current interface reference rather than any course text before writing the call, since the fixed-budget form is already being superseded by a mode where the model decides how much to think under an effort setting.

The second is agents that use a computer the way a person does: read the screen, decide on an action, execute it, look again. The point of it is legacy integration, because it works against any interface a human can operate, including systems that will never have a programmatic one. The limits are what decide whether you can deploy it. Error rates climb on complex interfaces with many controls, modal dialogues and content that changes between screenshots. Every step costs a screenshot, an inference and an execution, so latency accumulates step by step. The security boundary is unusually wide, because whatever is visible is available to the agent, credentials included, which makes strict sandboxing a precondition rather than a hardening step. And the same instruction may take different routes on different runs, so verify the resulting screen state after each action rather than assuming the click landed.

The third is agents that write code with access to a file system, a terminal and a test runner, iterating on test results rather than producing one response. The guideline the module gives generalises well beyond coding: use them where you can write a clear definition of done and verify the result programmatically, and stay cautious where success is a matter of judgement about an existing system. Beyond these, the module points at three directions to watch. Interoperability is layering: one open protocol standardises how an agent reaches tools and data, and a second standardises how agents delegate work to each other across organisational and vendor boundaries, with governance of the latter now sitting with a foundation and a multi-vendor steering committee rather than a single company. Long-horizon task completion targets runs of hundreds of steps with persistent state, where the open problems are error recovery deep into a run, context management and cost control. And training against a specified set of principles with model-generated feedback aims to align agent behaviour at a scale that human labelling of every example cannot reach.