Practice and strategy stage summary

An ML pipeline is not a script. It is a directed acyclic graph of dependent steps, each of which has to complete before the next can start: ingestion pulls raw data from source systems, validation checks schema and statistical properties, feature engineering turns raw data into model inputs, training fits parameters, evaluation measures performance against held-out data, and serving exposes the result to production traffic. Each step fails independently and for different reasons. Data sources change schema. Feature distributions move. Training jobs exhaust memory. Evaluation thresholds are breached. A system that handles each of those with retries, alerts and fallbacks degrades loudly; one that does not corrupts quietly, which is far more expensive.

The framing that makes this stick is boundaries rather than boxes. Draw the system as concentric rings and a request crosses all of them before it reaches the model, which means the audit trail, the escalation path and the named owner drawn on the outer ring cover every inner control, the model included. That is why the 2015 NeurIPS paper on hidden technical debt in machine learning systems is the reference the module leans on. The module quotes the widely repeated estimate that model code is around five percent of a production system; treat that as a rule of thumb rather than a measured constant, because the paper's own abstract makes the durable claim without a percentage. What it names are the ongoing maintenance costs: boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and system-level anti-patterns.

The practical consequence is a habit rather than a diagram. Before you accept a model into production, ask which ring each control sits in and who owns it. If the answer for logging, escalation or rollback is that it lives inside the model service and is owned by whoever trained the model, the outer rings do not exist yet and there is nothing between a bad prediction and a customer.

Platform components are easy to dismiss as big-company machinery, so it helps to name the specific failure each one removes. A feature store is a central repository for storing, versioning and serving engineered features, and the failure it removes is training-serving skew: features computed one way in a batch training job and a slightly different way in the real-time serving path, producing a model that evaluates well offline and disappoints in production. That failure is silent, hard to diagnose and independent of team size, which is why the module rejects the idea that feature stores are only worth it at scale. Define the feature once, serve it to training and inference through the same interface, and the class of bug disappears rather than being debugged.

An experiment tracker removes irreproducibility. Model development is genuinely experimental, with hundreds of runs across different hyperparameters, feature sets and data subsets, and without systematic logging of parameters, metrics and artefacts the team cannot answer which run produced the best result or reconstruct it. A model registry removes untracked deployment: it is a versioned catalogue with metadata about who trained each version, on what data, with what evaluation results and at what deployment stage, so promotion becomes a governed state transition rather than someone copying a file to a server and hoping. Orchestration is the one genuine choice among these, and the module's rule is to match the tool to the team rather than to the ambition: a Python-strong team running batch workloads is productive on Airflow immediately, a Kubernetes-native environment gets container-level reproducibility and GPU allocation from Kubeflow Pipelines, and Prefect sits between the two for teams that want scheduling without operating a cluster.

Monitoring is the component that keeps the rest honest, because a model that passes evaluation does not stay good. Data drift means the input distribution has moved away from the training distribution. Concept drift means the relationship between inputs and outputs has changed, which is the harder case because the inputs can look entirely normal. At a minimum, watch the distribution of each input feature and the distribution of the model's predictions, and treat a significant shift in either as a trigger to investigate rather than a chart to admire.

Every parameter occupies memory, and the width you store it at decides how much. Full precision floating point uses four bytes per parameter, half precision two, eight-bit integers one, and four-bit integers half a byte. Multiply parameter count by bytes per parameter and you have the weight footprint before anything else. Training needs roughly four times that, because you hold weights, gradients, optimiser state and activations at once, which is why distributed training across several accelerators is normal for large models. Inference needs roughly the weights plus a modest allowance for activations and cache. That asymmetry between training and inference memory is the foundation of every cost strategy that follows, and the arithmetic alone rules deployments in or out before any benchmark is consulted.

Three compression routes attack the footprint from different directions and they are not interchangeable. Quantisation lowers numerical precision and leaves the parameter count alone; post-training quantisation applies it to an already-trained model, while quantisation-aware training simulates the reduced precision during training so the weights adapt to it, which preserves more accuracy at the cost of retraining. Distillation trains a smaller student model on a larger teacher's output probabilities rather than on hard labels, so it reduces parameter count rather than precision, and the soft labels carry information about which classes the teacher considers similar that hard labels discard. Pruning removes what contributes little, and here the structured and unstructured distinction decides whether you get a real speedup: unstructured pruning zeroes individual weights and can reach high sparsity, but standard hardware gains nothing from scattered zeros, whereas structured pruning removes whole neurons, attention heads or layers and produces a model that is genuinely smaller on ordinary hardware.

Edge deployment is where the three routes have to be combined rather than chosen between. Running inference on a phone, a browser or a sensor removes network latency and per-request API cost and keeps data on the device, and the price is a hard memory and compute ceiling that no single technique clears on its own. The module's hybrid pattern is worth carrying: a small fast model handles common cases locally and difficult inputs are forwarded to a larger model elsewhere, which lowers average latency and cost while keeping quality where it matters.

Training is expensive once, or on a schedule. Inference happens on every user request for as long as the product exists, which is why inference dominates total cost of ownership for most production systems. The estimating formula the module teaches is simple enough to run in a meeting: cost per request is accelerator cost per second multiplied by latency per request, divided by batch size. What matters is not the number you get but the sensitivity it exposes. Batch size sits in the denominator, so serving one request at a time on hardware billed by the second is the most expensive way to use it, and the difference between a naive configuration and an optimised one is routinely more than an order of magnitude.

The levers are batching, caching, model routing and the compression techniques above. Batching raises utilisation of hardware you are already paying for. Caching removes repeated work for repeated inputs. Routing sends easy queries to a small model and reserves the large one for queries that need it. Compression lowers the per-request cost of the model itself. All four are multiplicative rather than alternatives, which is why cost work usually starts with the cheapest of them rather than with a model change.

The trap is generalising a measured cost from one configuration. Break a single inference into its parts and a short prompt is dominated by input and output tokens, so trimming the prompt genuinely helps. A retrieval-augmented request adds context retrieval and embedding refresh on top, and those two are frequently the largest blocks, so the same prompt-trimming exercise barely moves the bill. Quote a cost per request together with the precision, the batch size and whether retrieval was in the path, or it is not a number anyone can act on.

Precision, recall and cross-validation assume a clear ground truth. Generated text, code and images do not have one, so evaluation becomes a question of which evidence you can afford and what each kind of evidence cannot tell you. Human judgement is the reference standard: a person can say whether a summary caught the point, whether code is idiomatic, whether a translation kept the nuance. A rigorous protocol needs a written rubric, several independent annotators per item so that agreement can be measured with Cohen's kappa or Krippendorff's alpha, randomised presentation order, and a sample large enough to support the conclusion. The limitation is printed on the tin: it is slow, costly, and not reproducible across annotator pools, so it belongs at release decisions and safety assessments rather than in the daily loop.

Automated judging fills the daily loop. An LLM-as-judge receives the prompt, the response and a rubric and returns a score or a pairwise preference, at a speed and price that human panels cannot match, and the research behind it reports strong judges matching human preferences at over eighty percent agreement, which is about the level humans agree with each other. Its limitations are well documented and all three are correctable: position bias, where the response presented first is favoured, is handled by randomising order and running each pair both ways; verbosity bias, where longer answers score higher without adding substance, is handled by rubric design and length controls; self-enhancement bias, where a judge favours outputs from its own family, is handled by using more than one judge and calibrating against held-out human annotations. Treat it as a regression detector, not as ground truth.

Static benchmarks and adversarial testing complete the set. Benchmarks are useful for shortlisting and they saturate: as models improve they crowd the top of the scale and stop discriminating, and a high score on a multiple-choice suite says nothing about open-ended generation in your domain. Red teaming is the deliberately adversarial stream, combining automated adversarial prompt generation with human domain expertise, because a clinician spots subtly dangerous advice that an automated scorer will pass. Elo ratings from head-to-head leaderboards rank models on aggregate preference over that leaderboard's prompt distribution, which is why they are a discovery tool and not a deployment decision. Wire all of this into an evaluation harness that a release has to pass, so that promotion is gated by evidence in the same way a deployment is gated by a test suite.

The line between a workflow and an agent is where control flow is decided. In a workflow, models and tools are orchestrated through predefined code paths that a developer wrote. In an agent, the model decides dynamically what to do next based on what has happened so far, and keeps deciding until a goal is reached. That loop is both the capability and the risk: it lets the system decompose a task, gather information and adapt when something fails, and it lets each step's mistakes propagate into the next without the system noticing it has gone off track. The ReAct pattern is the readable form of the loop, interleaving a Thought that states the reasoning, an Action that calls a tool, and an Observation that returns the result, so that the next thought is conditioned on real data rather than on the training distribution. It outperforms reasoning-only prompting on anything that needs external information, and it is bounded by the tools available: an agent with search but no calculator will still get arithmetic wrong after finding the right formula.

Function calling is the mechanism underneath. The model emits a structured object naming the function and its arguments rather than free text that a parser has to interpret, the runtime executes it, and the result returns as a new message. Reliability therefore turns on schema design more than on model choice: precise tool names, descriptions that say when to use the tool and not only what it does, typed parameters with their own descriptions, and explicit error formats. Ambiguous descriptions cause wrong tool selection; missing parameter descriptions cause wrong arguments. The Model Context Protocol standardises this interface as an open standard for connecting AI applications to external systems such as data sources, tools and workflows, with servers exposing capability and clients consuming it, so that adding a ninth integration does not mean writing a ninth bespoke adapter.

Two disciplines keep agents from being demos. The first is stopping: the states of the loop close into a ring with no exit of their own, so the only reliable branch out runs through a budget gate on turn count and token spend, which means the run ends because a limit was reached rather than because the agent decided it had finished. The second is restraint about architecture. Multiple agents earn their coordination overhead, added latency and distributed debugging only when the task needs fundamentally different reasoning strategies or exceeds what one context can hold; different data sources are what tools are for. The published guidance is to start with simple prompts, evaluate them properly, and add agentic complexity only when it demonstrably improves outcomes.

Reinforcement learning has three elements and one loop. An agent observes the state of an environment, chooses an action, receives a reward and moves to a new state, and its goal is a policy, a mapping from states to actions, that maximises cumulative reward. Formalised as a Markov decision process, that is a set of states, a set of actions, a transition function giving the probability of the next state, and a reward function over state-action pairs, with the Markov property saying the future depends only on the current state and not on the path taken to it. The discount factor decides how far ahead the agent looks: near zero it is myopic and takes the immediate reward, near one it will accept a short-term loss for a later gain. This is why reinforcement learning suits problems where the signal is the consequence of a sequence of decisions rather than a label attached to an example.

Two families of method sit on top of that frame. Q-learning estimates the value of taking an action in a state and then behaving optimally, learns purely from experience without a model of the environment, and works when the action space is discrete and small enough to enumerate; replacing the lookup table with a neural network is what let it work on high-dimensional inputs. Policy gradient methods learn the policy directly as a parameterised distribution over actions and therefore handle continuous action spaces that Q-learning cannot, at the cost of noisy estimates, which is what the clipping in Proximal Policy Optimisation exists to tame. Cutting across both is the exploration and exploitation trade-off: exploit and you may settle in a local optimum, explore and you spend effort on actions you already suspect are worse. Epsilon-greedy with a decaying epsilon is the baseline, and the same tension shows up in A and B testing, clinical trial allocation and portfolio choice.

RLHF rewires this loop for language models in three stages: supervised fine-tuning on demonstrations of the behaviour you want, a reward model trained to predict human rankings of model outputs, then policy optimisation against that reward model. A KL penalty keeps the optimised policy close to the fine-tuned checkpoint, and its purpose is specific: without it the policy drifts into regions the reward model was never trained to judge and starts scoring well on outputs that are degenerate. That is the same phenomenon as reward hacking, which is not an algorithmic bug but the predictable result of optimising a proxy. The reward function is a proxy for what you wanted; if the proxy has a gap, optimisation finds it. Engagement metrics that reward outrage, and helpfulness rewards that produce agreeable, confident and wrong answers, are the production versions of the same failure.

Three developments define the current frontier and each has a precise technical content that is easy to overstate. Multimodal models process text, images and audio inside one architecture rather than chaining separate models together, and the mechanism is that every modality is turned into tokens: images into patches, audio into spectrogram-derived tokens, text as usual, all attended over together. What that buys is the preservation of cross-modal relationships that a pipeline discards at each handoff, such as tone of voice lost when speech becomes text or spatial layout lost when an image becomes a caption. Reasoning at inference time is the second. Asking a model for intermediate steps improves results on some tasks because each generated token is another forward pass and stays in context for the ones after it, and newer models can spend variable amounts of hidden reasoning before answering. Neither visible rationale nor hidden reasoning is proof that the stated reasoning is the actual cause of the answer, so ask for evidence, assumptions and checkable claims rather than treating a rationale as an explanation.

World models and emergence are the third and fourth. A world model is an internal representation of how an environment behaves, and the evidence that large models develop something resembling one is suggestive rather than settled. Emergence, the claim that abilities absent in smaller models appear suddenly at scale, is genuinely contested: the counter-argument is that discontinuous, all-or-nothing metrics manufacture the appearance of a jump, and that continuous metrics show smooth and predictable improvement instead. For anyone planning, the difference matters. If capability arrives unpredictably you cannot forecast the next generation; if it scales smoothly you can. The current position is a mixture, so plan with explicit uncertainty ranges rather than point predictions.

The habit that survives all of this is a sorting rule. Place a claim by the quality of the evidence behind it and by whether it changes a decision you are actually making. Promote it rightwards only when the evidence replicates, demote it when its relevance to your decisions fades, and never let it move because the demonstration was impressive. The agent story from Module 20 is the worked example: curated demonstrations and systematic benchmarks told different stories about the same system, and reporting the benchmark number without naming which variant of the benchmark it came from would have been almost as misleading as the demonstration.

The alignment problem splits cleanly and both halves are open. Outer alignment asks whether the objective is the right one: tell a model to be helpful and it can become sycophantic, tell it to be harmless and it can refuse anything that could conceivably be misused, and in both cases the optimisation worked and the specification did not. Inner alignment asks whether the model actually pursues the specified objective or has learned an internal objective that merely correlated with the training signal and diverges in situations the training never covered. Interpretability is the discipline that would let you check. Mechanistic interpretability reverse-engineers circuits and has identified components that implement recognisable functions; probing trains small classifiers on internal representations to test what information is present; attention analysis shows what the model treats as relevant, which is diagnostic rather than causal. The honest summary is neither total opacity nor understanding: it is partial and improving, at a scale far short of a full account of any frontier model.

Two techniques respond to the fact that human oversight does not scale. Constitutional AI gives the model a written list of principles and trains it in two phases: a supervised phase in which the model critiques and revises its own outputs against those principles and is fine-tuned on the revisions, then a reinforcement learning phase that uses AI feedback as the reward signal, with human oversight entering only through the principles themselves. The wider problem it addresses is scalable oversight: a person can judge whether a response is polite, but not reliably whether an analysis of a long technical document is correct, and that gap widens as capability grows. Safety evaluation then has to run as a loop rather than a gate, with four stages that each produce a named artefact and pass it on as findings, a trained model, a deployed system or incidents; a stage that writes nothing down leaves the next stage with no input.

Governance sits outside the loop and differs by jurisdiction. The EU AI Act classifies systems by risk, with unacceptable-risk uses prohibited, high-risk uses subject to strict requirements, limited-risk uses carrying transparency obligations and minimal-risk uses unconstrained, and hiring, credit scoring, criminal justice and critical infrastructure fall in the high-risk band. The United Kingdom put its weight behind evaluation rather than prescriptive rules, through a national institute that now operates as the AI Security Institute, whose stated mission is to equip governments with a scientific understanding of the risks posed by advanced AI and whose work covers monitoring AI development, evaluating risks to national security and public safety, and advancing safeguards, alignment and control. The United States route has run mainly through executive action and voluntary industry commitments. The existential risk debate that sits behind all of this has three positions worth engaging with rather than dismissing: that the risk is real and near, that it is real but distant while present harms are documented and immediate, and that the risk discourse itself diverts attention from those present harms. Defence in depth, several independent layers none of which is relied on alone, is the practical response whichever position you hold.

Before any use case is chosen, assess what each part of the organisation can actually absorb, across four dimensions. Data readiness asks whether clean, accessible data and working pipelines exist, or whether they would have to be built. Technical infrastructure asks whether the compute, storage and operational tooling are there, and whether constraints such as data residency force an on-premises deployment. Human capability asks whether anyone can build, evaluate and maintain the system after the consultants leave. Cultural receptivity asks whether leadership will change workflows and accept model-assisted decisions. Skipping this step is how programmes end up starting with the most technically interesting use case rather than the one most likely to succeed.

Prioritisation then runs on two axes, business impact and implementation feasibility, and the useful output is clusters rather than a ranked list. Quick wins combine high feasibility with moderate impact and exist to produce real performance data and organisational confidence. Strategic bets combine high impact with moderate feasibility and need planning, usually because the data is sensitive or the regulator requires explainability. Foundations, such as a unified data platform, enable everything later while delivering nothing directly, which makes them essential and hard to fund. And some things are correctly deferred: impact does not rescue a use case whose cost, timeline and risk exceed the budget and the window. Build or buy follows from the same constraints rather than from preference, since an API-hosted model is faster for general tasks while an open-weight model on your own infrastructure may be the only option when data cannot leave the organisation, and a vendor model has to be evaluated independently on the population it will actually serve.

Governance is designed in, not added afterwards, and in a regulated setting it is not optional. Every production model needs a model card documenting intended use, training data, evaluation results, limitations and known failure modes, in the form the original proposal set out: benchmarked evaluation across relevant demographic and intersectional groups, the context the model is intended for, and the evaluation procedures used. Fairness testing has to be per subgroup rather than in aggregate, human oversight has to give a person the ability to review and override an automated decision, and incident response needs an escalation path and a rollback procedure written before it is needed. The launch gate that ties this together is an evidence pack: nine artefacts grouped under three accountable audiences, engineering, compliance and leadership, each card naming the artefact, the question it answers and the owner who signs it. The gate needs all nine present and signed, so the column that is ready first counts for nothing on its own.