Stage 2 summary. Applied systems and AI

Bronze, silver and gold is a discipline about where truth is allowed to change. Bronze holds raw arrivals exactly as received, so the pipeline can always be replayed. Silver holds cleaned, conformed and deduplicated data with the transformation rules written down. Gold holds the shaped, aggregated products that consumers actually query. Keeping the three separate means a quality dispute can be resolved by inspection rather than by argument, because the layer where a value changed is visible.

ETL and ELT differ in where the transformation cost lands. Transforming before loading limits what reaches the warehouse and controls storage cost, at the price of having to know the questions in advance. Loading first and transforming in the warehouse keeps the raw record available for questions nobody has asked yet, at the price of paying for compute and storage on data that may never be used. Change data capture reads the source's own change log rather than repeatedly scanning tables, which is what makes near-real-time feeds affordable, and orchestration decides whether a failed step blocks, retries or degrades.

A data contract makes the interface between producer and consumer explicit: schema, semantics, freshness, volume expectations and what happens when they are breached. The Open Data Contract Standard gives a published shape for writing one. The paired service level agreement needs two parts to be real, a stated commitment and a named consequence, because an agreement with no consequence is a preference. On data mesh, the 2026 consensus is that the domain ownership and data-as-a-product principles have held while full decentralisation has not, so most organisations run federated ownership over shared central infrastructure.

A north-star metric names the single quantity that best represents delivered value, and its job is to be the thing input metrics are argued against. Input metrics are the levers a team can actually move within a quarter, and the relationship between them and the north star is a hypothesis to be tested rather than a diagram to be admired. When a team cannot say which input metric its work moves, the work has no measurement story.

Three categories get conflated and should not be. A performance indicator measures how well something is being done. A risk indicator gives early warning that something is becoming more likely to go wrong, so it leads rather than lags. A vanity metric moves reliably upwards, feels good, and changes no decision, with cumulative registrations the classic example. The test is whether a change in the number would change what anyone does next.

Experiments and funnels are where honesty is tested. An experiment needs its success measure, its population and its stopping rule fixed before it runs, because a stopping rule chosen afterwards guarantees a result. A digital twin belongs in this section rather than in the technology one, because the framing that makes it useful is the control loop: a model kept in step with a physical asset by live data, used to decide an intervention, with the outcome fed back.

REST suits resource-shaped, cacheable, widely consumed interfaces where the consumer set is unknown. GraphQL suits clients that need to assemble varied shapes from one round trip, and moves query cost onto the server, which then needs depth and complexity limits. gRPC suits high-volume service-to-service calls where a compact binary contract and streaming matter more than being readable in a browser. The choice follows who is consuming and under what constraints, not which style is newest.

Versioning is the promise that consumers can keep working. Additive change that leaves existing fields and behaviours intact does not need a new version. Anything that removes or repurposes what a consumer already relies on does, and it needs an overlap period long enough for consumers to move. Publishing the contract is what makes this checkable, and the current specification versions matter: OpenAPI 3.2.0 for request and response interfaces, AsyncAPI 3.1.0 for message-driven ones, and CloudEvents 1.0.2 for describing an event's metadata consistently across transports.

Event-driven design decouples producers from consumers in time and identity, and buys three problems in exchange. Ordering is no longer guaranteed unless it is designed for. Delivery is usually at-least-once, so consumers must be idempotent. And the system's behaviour becomes emergent, which is why tracing matters more here than in a request-response design. The Model Context Protocol is worth placing precisely in this section: it is an open protocol using JSON-RPC 2.0 that standardises how applications expose resources, prompts and tools to language models, which is an integration contract, not a governance regime and not a safety guarantee.

A capability is a stable statement of what an organisation is able to do, such as settle a payment or connect a customer. A process is one way of doing it, and it changes often. A function is a group of people, and it changes for reasons that have nothing to do with either. Keeping the three apart is what lets a capability map stay useful across a reorganisation, and it is why a map that mirrors the org chart has usually been drawn wrong.

Heat-mapping puts strategy onto the map. Each capability is scored for how much the strategy depends on it and for how well it currently performs, and the investment conversation goes to the cells that are strategically important and weak. A value stream reads the same organisation the other way, following one unit of work from trigger to delivered outcome and recording where it waits. Most delivery time in most organisations is waiting rather than working, so the handovers and queues are where the recoverable time sits.

TOGAF supplies the method around these artefacts rather than replacing them. The architecture development method gives a phased cycle from architecture vision through the business, information systems and technology architectures to migration planning and change control, and its building-block idea separates what a capability needs from the product that eventually provides it. A target operating model ties the set together, stating the capabilities, the value streams, the organisation that runs them and the technology underneath.

A service level indicator is the measurement of something a user actually experiences, such as the proportion of requests served successfully within a latency threshold. A service level objective is the target for that indicator over a window. The error budget is the difference between the objective and perfection, and it converts an argument about whether to ship into an arithmetic one: budget remaining means change can continue, budget exhausted means reliability work takes priority. Setting an objective at total availability destroys the mechanism, because a budget of zero can never inform a decision.

Instrumentation rests on three signal types. Metrics are cheap aggregates good for alerting, logs are detailed records good for reconstructing a specific event, and traces follow one request across services and are the only one of the three that explains where distributed time went. OpenTelemetry matters because it standardises the generation and export of all three, so instrumentation is not rewritten when the backend changes. Incident management works when it is blameless, and the reason is practical rather than pastoral: an engineer who expects to be punished withholds the detail that would prevent a recurrence.

The DORA delivery metrics have moved on from the original four keys to a five-metric model covering change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. The 2025 research adds the argument that AI functions as an amplifier of an organisation's existing strengths and weaknesses, with the largest returns coming from the surrounding system rather than the tools. Two different things share the name DORA, and they are never interchangeable: the DevOps Research and Assessment programme described here, and the European Union's Digital Operational Resilience Act covered in Stage 3.

A foundation model is a large general model adapted to many tasks rather than trained for one. Three adaptation routes cover most needs and they answer different questions. Prompting changes the instruction and the examples, and it is the right first move because it is reversible in seconds. Retrieval augments the prompt with material fetched from your own corpus at request time, and it is the right answer when the gap is knowledge the model never had or knowledge that changes. Fine-tuning adjusts the model's weights, and it is the right answer when the gap is form, style or a task shape that instructions keep failing to convey. Cost, latency and the ability to explain an output all move differently across the three.

A copilot suggests inside a workflow a human is driving, and the human remains the point of control. An agent plans and executes multi-step work against real systems, and control has to be designed in explicitly: a bounded set of tools, authorisation checked at the point of action rather than at the start of a session, a human approval gate on consequential steps, a spend and step budget, and a log good enough to reconstruct what was done and why. The Model Context Protocol standardises how tools and data are offered to a model, which makes integration portable but leaves every one of those control points as the implementer's responsibility.

Governance has a shape. The EU AI Act defines four risk tiers, from unacceptable practices that are prohibited, through high-risk uses carrying obligations for risk management, data quality, logging and human oversight, through transparency-risk uses where people must be told they are dealing with a machine, to minimal risk with no specific rules. It entered into force on 1 August 2024, its prohibitions applied from 2 February 2025, its general-purpose model and governance provisions from 2 August 2025, and the bulk of the remaining obligations from 2 August 2026, with certain high-risk categories following in December 2027 and August 2028. The UK has taken a regulator-led route rather than a single statute, applying existing sectoral regulators to AI use within their remits. Management-system anchors such as ISO/IEC 42001 give an organisation an auditable frame either way, and none of it compensates for weak process or untrusted data.