Module E1 of 3 · Exam and certification

Course revision guide

45 min 6 stages distilled 6 figures

The whole course in one place, arranged for revision. Each stage below gives you its figure, the distinctions worth holding in your head, and the fastest routes back into the full material. Skim the figures first, then test yourself against the distinctions: anything you cannot explain from memory points at the stage to revisit.

Stage 1: What an agent is, and the environment it runs in

AI is pattern-based capability, not a mind

Capability rises with the data each class learns from, but no row in that stack says whether an output is usable. The four NIST AI RMF 1.0 gates below settle that: measure error, check deployment context, manage residual risk, document limits. A good error score is not clearance.

Capability comes from pattern recognition over data. Usability comes from verification, not from the model. NIST AI RMF 1.0 names verification as the deployment gate.

Hold these distinctions

Artificial intelligence versus machine learning versus deep learning: concentric rings, not synonyms. The outer ring includes rule-based systems that never learn anything, the middle ring infers patterns from examples instead of hand-written rules, and the inner ring stacks many layers to learn hierarchical representations
Rule-based versus learned: a rule is auditable and brittle, and you can read exactly why it fired; a learned model tolerates unanticipated input and cannot tell you why it produced a given answer. That trade decides which one belongs in a given part of a system
Narrow AI versus artificial general intelligence versus superintelligence: the first is built for a specific task, can be superhuman at it and useless outside it, and is what every deployed system is; the second and third are hypothetical, and conversational breadth is evidence of training breadth rather than of generality
Token versus word: models operate on sub-word units, so cost, context limits and rate limits are all counted in tokens rather than in words, and a rare term consumes more of the budget than its length suggests
Context window versus memory: the window is how much can be supplied in one call, including the system prompt, the history and every tool result. It is not recall, and the interface retains nothing between calls
Large language model versus chatbot versus agent: a model maps text to text, a chatbot wraps that in a conversational interface, and an agent adds tools, state across steps and a loop with a stop condition. Only the third can act on the world
Model versus application layer: the model emits a structured request naming a tool and its arguments, and your code executes it. The model never reaches the network or the database itself, which is why validation, error handling and result structure are your obligations
Thought versus action in the ReAct pattern: the thought is a recorded reason that makes a failure traceable to a specific decision, and the action is the call that changes something. A fluent thought is not evidence that the answer is right
Session environment variable versus shell profile entry versus .env file: export sets a value until the terminal closes, a line in ~/.zshrc or ~/.bashrc persists it for every future session, and .env scopes it to one project and is loaded by the application. Only the last belongs in a project directory, and only if it is in .gitignore
Python virtual environment isolation versus Node module isolation: Python isolates by giving the project its own interpreter environment that must be activated, Node isolates by installing dependencies into a directory inside the project. Both keep one project from breaking another, by different means
The environment directory versus the file that describes it: .venv and node_modules are disposable and platform-specific and should never be committed; requirements.txt and package-lock.json are the artefacts that let the environment be rebuilt, and they must be
Input tokens versus output tokens: both are billed and both are reported in the response usage object. In a loop the input side grows fastest, because the system prompt, the whole conversation and every accumulated tool result are resent on every step
Low versus high sampling temperature: low is correct for anything another program will parse, where variation shows up as malformed structured output; higher settings belong only in prose a person will read. Set the value explicitly rather than inheriting a default
Stateless interface versus stateful application: the service keeps nothing between calls, so the conversation exists only because your code resends the full message list each time. That gives the application complete control over what the model sees and puts the responsibility for state squarely in your code

Full stage summary Scenario practice

Stage 2: The loop: how an agent reasons, acts and remembers

ReAct makes reasoning and acting inspectable

Plan, action, observation and decision close into a cycle that ends only where the four stop conditions say: step cap reached, budget exhausted, goal answered, or confidence too low. The trace panel logs every action and result, and a loop without it cannot be tested.

ReAct exposes the reasoning loop. Plan, action, observe, decide each leave an inspectable record. OpenAI Agents SDK tracing operationalises this.

Hold these distinctions

Stateless loop versus persistent memory: the model carries nothing between iterations and re-reads the entire context window each time, so an agent's knowledge on any given turn is exactly what was injected into that turn and nothing else
Model versus application: the model decides what to call and with what arguments, the application decides whether the call is permitted, validates it, executes it and chooses what to inject back. Every control worth having lives on the application side
ReAct versus Plan-and-Execute versus Reflection: interleave reasoning and action when each step reveals what the next one needs, plan first and then execute when the steps are knowable upfront, and generate then critique then revise when quality matters more than latency
Reasoning quality versus reasoning quantity: a long fluent chain is an input to the answer, not evidence about it, and self-critique cannot reliably catch errors produced by the same reasoning patterns that generate the critique
Tool name versus tool description: the name identifies, the description routes. Selection accuracy is a function of whether the description says what the tool returns, when to call it and when not to, and it degrades as the tool set grows under a fixed attention budget
Schema validation versus semantic validation: a schema constraint catches structural mismatches such as a wrong type or a missing required field, a runtime validator catches the errors a schema cannot express, such as a well-formed string that names a resource which does not exist. Both layers are needed
Structured error versus raised exception: an exception ends the loop and the model never sees it, a structured error returned as a tool result keeps the loop alive and gives the model enough to retry, switch tools or report the failure
In-context versus external versus semantic memory: immediate but temporary and finite, persistent and exact-match by key, persistent and approximate by meaning. Choose on access pattern, not on data volume
Ingestion versus retrieval: ingestion chunks and embeds the corpus and runs when the corpus changes, retrieval embeds the query and searches and runs on every request. The same embedding model must be used on both sides or the vectors do not share a space
Retrieval quality versus generation quality: retrieval decides whether the right material was found, generation decides what was done with it, and an answer can be fluent and wrong because retrieval succeeded on an ambiguous query. Evaluate them separately
Sliding window versus summarisation: a sliding window discards early messages entirely and cheaply, summarisation compresses them and loses selectively, characteristically the constraint stated once at the very start. Keep recent messages verbatim in both cases and never summarise the system prompt
Router versus supervisor: a router classifies an incoming request and hands it to one specialised agent, a supervisor decomposes work across several sub-agents and synthesises their results. The router is a classification problem, the supervisor is an orchestration problem with a hallucination-laundering hazard
Map-reduce versus chain: map-reduce applies the same operation across many independent items and then aggregates, a chain applies different transformations in sequence where each step consumes the previous step's output. The first is about volume, the second is about stages
Agent state versus conversation history: the orchestration code reads state to decide which node runs next, the model reads history to decide what to do next. Conflating them in one abstraction makes a single point of failure for both
Conditional edge versus checkpoint: a conditional edge is routing, deciding which node runs next by reading current state, a checkpoint is persistence, saving state at transitions so a workflow can resume after interruption and so a person can inspect and modify it at an approval gate
Independent versus dependent operations: independent operations can run concurrently and cost roughly the slowest of them, dependent operations must stay sequential because one supplies the other's arguments. The dependency graph decides this, not the desire for speed

Full stage summary Scenario practice

Stage 3: Building one, then more than one, and connecting them to real systems

A first agent is a small loop with hard boundaries

The first tool is read-only and the loop runs under a hard cap, so a wrong first agent costs a wasted run rather than a change you have to undo, and the test set built at the fourth step is the part of it that survives the rewrite.

A first agent is a small loop with one safe tool, a hard step cap, and tests for both happy and bad inputs. ReAct and the OpenAI Agents SDK quickstart name this contract.

Hold these distinctions

Tool description versus system prompt: the prompt is read once at the top of the conversation, the description is read every time the model weighs that tool, which is why wrong tool selection is fixed in the description first
Stop reason versus step limit: the loop's three states are a final answer, a request for tool calls, and the hard maximum being reached. The first two are the agent working; the third is a safety belt, and hitting it regularly is a symptom to diagnose rather than a number to raise
Unit test versus integration test: tools are tested in isolation first, so that when the whole-loop test fails you already know the fault lies in the loop, the schemas or the system prompt rather than in a tool function
Single agent versus multi-agent: one agent is simpler and has no handoff to lose context across; several agents buy narrow tool sets, focused contexts and contained failures, and pay in tokens, latency and new coordination failure modes
Supervisor versus worker: the supervisor plans and synthesises, which is where reasoning quality matters, while workers execute bounded, defined tasks, which is why matching model capability to the job rather than using one model everywhere is the standard economy
Structured handoff versus prose instruction: a handoff carries the sub-task, the context, the expected output format and a fallback instruction, so the receiver has nothing to guess at and the supervisor parses the reply instead of interpreting it
Sequential versus concurrent workers: work that is genuinely independent should run at the same time, so three tasks of equal length cost about one task's time plus coordination rather than three times it. Work with real dependencies gains nothing from concurrency
MCP host versus client versus server: the host runs the model, the client is the protocol layer inside it, and the server exposes capabilities and executes them with its own credentials, which is why the model never sees the database password
MCP tool versus resource versus prompt: tools are functions the model can execute and may have side effects, resources are context and data to read, and prompts are templated workflows a user can invoke. The distinction is what tells a reviewer what the agent can change as opposed to only read
Local subprocess transport versus networked transport: the first is secured by process isolation and suits single-user tooling on one machine, the second exposes an endpoint several clients can reach and is only secure with encryption and authentication. Choose by who has to reach the server, not by which sounds safer
Status code versus response validity: a successful status does not mean a successful operation, since query language interfaces routinely report failures in an errors field inside a successful response, so validate the shape of the body rather than the code alone
Transient versus permanent failure: rate limiting and a temporarily unavailable service may clear with exponential backoff and jitter, whereas a rejected credential will be rejected identically next time, so it is refreshed or replaced rather than retried
Exponential backoff versus a circuit breaker: backoff handles a single call that may succeed if it waits, the breaker handles a service that is failing consistently and stops calling it entirely until a probe succeeds
Error for the developer versus error for the agent: the developer needs the full exception in a server-side log, the agent needs a testable success flag, a category and a short plain-language description, and giving the agent the developer's version leaks internal detail it may pass on

Full stage summary Scenario practice

Stage 4: What can go wrong when a system acts, and the controls that bound it

Agent risk appears where instructions meet capability

Untrusted content, context confusion, capability misuse and data or action harm run in that order, and each stage names the one control that reduces it: hygiene, instruction hierarchy, tool allow-list, egress filter. Clean input does nothing about a wrongly granted tool.

Prompt injection cannot be removed by wording. Treat each stage as a control surface: context hygiene, instruction hierarchy, tool allow-list, output and egress filter. OWASP LLM 2025 and NCSC name these controls.

Hold these distinctions

Content risk versus operational risk: a model that writes something wrong is a correction and an apology, an agent that acts on something wrong may have sent, deleted or paid, and the threat model has to be built around consequences rather than around output quality
Direct versus indirect prompt injection: direct is typed into the interface by the attacker, indirect is planted in content the agent will retrieve during a legitimate task, so the attacker never touches the system and the material can be hidden from the human who would otherwise notice it
Prompt injection versus jailbreaking: injection redirects the agent's task towards the attacker's goal, jailbreaking strips the model's safety constraints so it produces output it was trained to refuse. Different targets, different mitigations, and both are arms races
Exfiltration through a tool call versus through rendered output: the first needs a permitted sending or requesting tool, the second needs only the ability to emit a URL that the interface will fetch, which is why restricting the tool list is not by itself a complete answer
Agent hijacking versus a compromised credential: hijacking plants instructions in a tool result and steers the agent mid-task using permissions it already holds, so authentication succeeds, the traffic looks correct and nothing at the infrastructure layer fires
Excessive agency versus every other risk: excessive agency causes no incident on its own and sets the price of all of them, which is why reducing the tool set is the highest-leverage single control in the stage
Input validation versus retrieved content sanitisation: the first checks what users type and is the easy half, the second cleans markup, hidden characters and over-long documents out of everything the agent reads, and is the half that is usually missing
Output schema validation versus content filtering: validating against a declared structure is checkable and repeatable, whereas trying to detect malicious intent in free text is not, which is why structure is the more reliable place to put the gate
Tool scope versus credential scope versus data scope: least privilege has to hold in all three at once, since the right tool list with an over-powerful credential, or the right credential with the whole document store in reach, leaves the exposure roughly where it was
Transparency versus explainability versus interpretability: transparency is telling people an AI system is involved, explainability is whether they can understand and check why a specific output was produced, interpretability is understanding the model's internals. A system can be fully transparent and completely unexplainable
Model-level safety versus system-level safety: training-time constraints reduce the likelihood of compliance with a harmful instruction, application-layer controls decide what the system can do when compliance happens anyway. Neither substitutes for the other
Provider obligations versus deployer obligations: building and placing a system on the market and putting somebody else's system to work in a particular context are separate roles carrying separate duties, and integrating a third-party model into a hiring or lending decision makes you answerable for that deployment whoever trained the weights
Voluntary framework versus binding regulation: a risk management framework or a management system standard structures the work and can evidence diligence, while a regulation sets obligations that apply whether or not you adopted either, so conformity to a framework is not by itself compliance with the law

Full stage summary Scenario practice

Stage 5: Tuning, running and observing an agent in production

Fine-tuning is a narrow optimisation, not a fact database

All three branches leave the same root, so the tagged reason a case failed is what picks between them, and fine-tuning a gap that retrieval would close buys a training run and a rollback plan for a problem the adapter was never going to fix.

Fine-tuning fixes style, format, and specialised behaviour. Retrieval fixes facts. Hugging Face PEFT and the OpenAI fine-tuning guide name the gates that hold every branch safe.

Hold these distinctions

Prompting versus fine-tuning: prompting is fast, cheap and reversible and is the correct default, while fine-tuning answers what prompting cannot, meaning a format that costs hundreds of instruction tokens per call, domain conventions the base model lacks, inference cost at high volume, or data that cannot leave your infrastructure
Full fine-tuning versus parameter-efficient fine-tuning: the first updates every weight and pays for gradients and optimiser state across all of them, the second freezes the overwhelming majority and trains a small set of added parameters, which is what brings the memory requirement inside a single card
Low-rank adaptation versus quantisation: adaptation is about where the trainable parameters live, namely two thin matrices beside a frozen weight matrix, while quantisation is about how cheaply the frozen part is stored. The quantised variant applies both and keeps the adapters at higher precision so training quality holds
Rank versus quality: rank sets the capacity of the correction, so it trades trainable parameters, memory and training time against how much adaptation is possible, with returns that flatten. It is a capacity knob, not an accuracy knob
Fine-tuning versus retrieval: training changes how the model writes and behaves, retrieval changes what evidence it has in front of it. Factual failures belong to retrieval, stylistic and structural failures belong to training, and production systems commonly need both
Training loss versus held-out task metric: the first says the optimiser is working, the second says whether the model is better at the job. Only the second can be compared against the base model and against a good prompt
Synchronous versus asynchronous handling: synchronous makes every caller wait for the work and lets one slow request delay the queue behind it, asynchronous returns an identifier immediately and lets independent workers absorb the load, which is what buys backpressure and redelivery
Traces versus logs versus metrics: a trace reconstructs one request as linked spans sharing a trace identifier, logs carry the detail for a step, and metrics aggregate across all requests. You need all three, and the trace identifier is what joins them
Median versus tail latency: the median describes the typical experience and the ninety-ninth percentile describes where the complaints come from. An average conceals both, and a tail problem is usually architectural rather than a matter of model speed
A guardrail in the prompt versus a guardrail in code: the first sits inside the context window alongside untrusted input and can be argued out of it, the second runs over the input and the output independently of the model and cannot
Alert threshold versus rollback trigger: the first is the level at which a person should look, the second is the level at which the system reverses itself without asking. Keeping them as one number produces either constant false rollbacks or none at all
Split testing versus blue-green deployment: the split test is a measurement device that answers whether the change is better, the blue-green pair is a safety device that keeps the previous version running so reversing is a routing change. They answer different questions and mature releases use both
Agent-to-tool versus agent-to-agent standardisation: one open protocol standardises how a single agent reaches tools and data sources, a second standardises how independent agents discover each other and delegate work across organisational and vendor boundaries. They are complementary layers, not competing options
A capability demonstration versus production readiness: a demonstration shows the capability exists under favourable conditions, production requires reliability across the real distribution of inputs, cost that survives volume, a security boundary you can state, and enough determinism to write a test

Full stage summary Scenario practice

Stage 6: Designing, reviewing and evidencing a build of your own

The capstone is an evidence pack, not just code

The four evidence cards and the three artefact bands beneath them answer to one closing criterion: a reviewer judges the project from the pack alone, without the builder in the room. Anything explained only in conversation is a gap in the pack, not a detail.

A reviewer should be able to judge a working agent from the evidence pack alone. The C4 model and NIST AI RMF name the artefacts the pack carries.

Hold these distinctions

Requirements versus architecture versus implementation: the cost of change rises sharply along that sequence, which is the whole argument for writing the charter and the diagram before the code rather than after it
Functional versus non-functional versus security requirements: the first carry an identifier, a priority and a verifiable acceptance criterion, the second carry a target and the method by which it will be measured, and the third name data classification, authentication and authorisation controls
The requirements list versus the out-of-scope list: the first says what the agent must do, the second is what stops it accumulating responsibilities that were never designed, tested or documented, and only the second is enforceable against scope creep
C4 Level 1 versus the levels below it: level one is the system context, showing the agent as one box among its users and external systems, while the C4 hierarchy continues through containers, components and code, so an internal view of the loop, its tools and its memory is a component view rather than a context one
The decision versus the consequences in an architecture decision record: the decision is what most records contain and the consequences are what a reviewer scores, because that is where the accepted cost is written down while changing your mind is still cheap
Happy path versus complete: running end-to-end on test inputs is the minimum threshold for implementation, while completeness means failure cases handled, limitations documented, permissions restricted to the minimum and an engineer who was not involved able to operate it
Unit versus integration versus edge-case testing: unit tests cover each tool on its happy path plus at least two error cases, one integration test exercises the whole loop on five representative requests, and the bias or edge-case test exists to document a known failure mode and its mitigation
Observation versus recommendation in review feedback: the observation names what was seen at a specific file and line, the recommendation says what to do about it and why, and feedback that merges the two is the reason a reviewer's comment gets read as an opinion
Self-assessment versus peer review: self-assessment catches the issues the author is able to catch, peer review catches what familiarity blindness hides, and the course requires both because neither is a substitute for the other
Certification versus competence: a certificate documents evidenced learning at a point in time, competence is demonstrated by repeated application, and conflating them is how a foundation gets treated as a finish line
Primary versus secondary trade-off: the scoring gives full marks only when the pattern is right and the primary trade-off is correctly identified, because naming the wrong cost is evidence that the pattern was matched rather than reasoned about
Security as an architectural input versus security as an add-on: least privilege, approval gates, audit logs and deletion requirements belong in the first diagram, since retrofitting them means redesigning the permission model, the logging schema and often the data storage layer

Full stage summary Scenario practice

Exam technique for the two papers

The Foundation paper is 40 single-answer questions in 60 minutes, closed book, and it tests precise recall. Revise the pairs that sound alike and are not: a against an agent, a tool name against a tool description, a schema check against a runtime check, in-context against external against semantic memory, direct against indirect injection, an MCP host against a client against a server. In the sitting, read the stem twice and decide what is actually being asked before looking at the options, because many questions turn on one qualifying word.

The options are written so that measuring them does not help. The correct answer is as often the short one as the long one, and no option is there to be eliminated for being obviously silly, so eliminate on meaning rather than on shape. A precise claim beats a sweeping one, and an option that would be true of a different mechanism is the commonest way to be wrong.

The Practitioner paper is 8 workplace scenarios in 90 minutes, open book, scored five, three, one and zero. Rank the options rather than hunting a single right answer. Across this course the five-point answer almost always puts the control on the application side and at the point of action: it names the the agent may reach, the credential it runs with, or the evidence a reviewer could check, rather than trusting an instruction in a prompt. The three-point answer is normally real work aimed slightly wrong, most often sequenced too early or answering a question nobody asked. Use the reference panel to confirm a distinction, not to learn the topic inside 90 minutes.

The primary sources behind the course

Version-sensitive claims across the course anchor to primary, dated sources, and every stage summary cites the ones its teaching rests on. They are collected here so revision starts from the authority rather than from a summary of it.

Yao, S. et al. ReAct: Synergizing Reasoning and Acting in Language Models (2022)
Anthropic Messages API reference
Python venv documentation
python-dotenv project README
OWASP Secrets Management Cheat Sheet
Turing, A. M. Computing Machinery and Intelligence, Mind, Volume LIX, Issue 236 (October 1950), pages 433 to 460
Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal (2024)
Mata v. Avianca, United States District Court for the Southern District of New York (2023)
Wei, J. et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022)
Wang, L. et al. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023)
Shinn, N. et al. Reflexion: Language Agents with Verbal Reinforcement Learning (2023)
Wu, Q. et al. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (2023)
Anthropic tool use documentation
JSON Schema specification
LangGraph documentation, overview
Python asyncio documentation
British Columbia Civil Resolution Tribunal, Moffatt v Air Canada (2024)
Reported disclosure of proprietary material to a public assistant, Module 8 opening incident
Anthropic Engineering, Building Effective Agents
Model Context Protocol specification
RFC 6749, The OAuth 2.0 Authorization Framework
OWASP API Security Top 10 (2023), API1 Broken Object Level Authorization
MDN Web Docs, Retry-After response header
Python documentation, asyncio coroutines and tasks
n8n documentation, AI Agent node
OWASP risk lists for large language model and agentic applications
n8n licensing terms
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T. and Fritz, M. Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023)
Code of Practice for the Cyber Security of AI, Department for Science, Innovation and Technology
OWASP Top 10 for Agentic Applications 2026, OWASP GenAI Security Project
OWASP Top 10 for Large Language Model Applications, project page
NIST AI Risk Management Framework
Bai, Y. et al. Constitutional AI: Harmlessness from AI Feedback (2022)
Docker Engine security documentation
Regulation (EU) 2024/1689, the Artificial Intelligence Act
ISO/IEC 42001:2023, Artificial intelligence management system
Equality and Human Rights Commission guidance on artificial intelligence and equality
Hu, E. et al. LoRA: Low-Rank Adaptation of Large Language Models (2021), arXiv:2106.09685
Dettmers, T. et al. QLoRA: Efficient Finetuning of Quantized LLMs (2023), arXiv:2305.14314
Hugging Face PEFT library documentation
Site Reliability Engineering: How Google Runs Production Systems, chapter on monitoring distributed systems
OpenTelemetry documentation, Traces
Kubernetes documentation, HorizontalPodAutoscaler
Docker documentation, Multi-stage builds
Anthropic documentation, extended thinking
A2A Protocol documentation
Wang, D. et al. DocLLM: A layout-aware generative language model for multimodal document understanding (2024), arXiv:2401.00908
ISO/IEC 42001, management systems for artificial intelligence
The C4 model for visualising software architecture
Architectural Decision Records, adr.github.io
Nygard, M. Documenting Architecture Decisions (2011)
Google Engineering Practices, Code Review Developer Guide
Principles of Chaos Engineering
ISO/IEC 42001:2023, the AI management system standard
CPD Standards Office, CPD Quality Mark criteria
Regulation (EU) 2016/679, the General Data Protection Regulation, Article 17
The Open Group, TOGAF Standard

When the distinctions hold from memory and the figures feel obvious, you are ready to drill the papers and then sit the mocks.

Start the exam practice Go straight to the mocks

Loading lesson...