AI-era Digitalisation

A large language model produces language by repeatedly predicting the next token. Its fluency comes from approximating the statistical distribution of the text it was trained on, and so does its capacity for confident error. The NIST Generative AI Profile, AI 600-1, names that failure mode confabulation, defined there as the production of confidently stated but erroneous or false content, known colloquially as hallucination or fabrication. The renaming earns its keep: the danger is not that the model is sometimes wrong, it is that wrong output arrives in exactly the same register as right output. The same profile notes that outputs may include confabulated logic or citations that purport to justify the answer, which is what makes a confident error harder to catch than a blank one.

A foundation model is one trained on broad data that can then be adapted to many downstream tasks. Adaptation is where practitioners get confused, because four different techniques are described with the same loose verb. Prompting changes the instruction and nothing else. Retrieval-augmented generation pairs the model with a separate retrieval system so the material the answer is built from is fetched at answer time rather than recalled from weights. Fine-tuningcontinues training from an existing checkpoint, so it changes the weights themselves. Pretraining builds a new model from random weights.

Sorting them by cost is the wrong first move. Sorting them by the failure you want to avoid is the right one. If the failure is that the system does not know something, fine-tuning is the wrong instrument, because retraining is a slow and expensive way to load a fact that a retrieval step could have supplied in milliseconds. If the failure is that the output arrives in the wrong shape, tone or format, retrieval will not help, because the material was never the problem. If the failure is that retrieval returns the wrong document, no amount of prompt engineering repairs it, because the model is answering faithfully from what it was handed.

Two further terms belong in the same conversation. An open-weight model is one whose trained parameters are published for download, which changes where the model runs and what you may do with it, though open weights are not the same as an open training process. A reasoning model is post-trained to work through a problem in steps before answering, which earns its extra tokens and latency on hard problems with a checkable answer and rarely on routine text. Neither is a governance answer. Both are choices about where cost and control sit.

A copilot drafts, summarises or suggests inside a tool somebody is already using, and a person commits the result. An agentchooses the order of its own steps and calls tools to carry them out, so the path is settled while the run happens rather than written in advance. Anthropic's engineering guidance, published in December 2024, draws the line in the same place and adds a useful middle term: workflows are systems where models and tools are orchestrated through predefined code paths, while agents dynamically direct their own processes and tool usage. Most systems that ship are workflows wearing the word agent.

The distinction matters because it moves the blast radius. A copilot's worst case is one bad suggestion, and a person is standing between that suggestion and any consequence. An agent's worst case is everything its tools can reach, and there may be no person in the loop at the moment the damage happens. That is why agentic automationcannot be governed by reviewing the plan: there is no fixed plan to review. It has to be governed at the boundary.

Five control points do that work. A tool allowlist names every tool a run may call, so anything unlisted fails closed. A human checkpoint is a named step where a person approves before an action lands. A stopping condition caps turns, spend and elapsed time, so a loop ends without a person noticing it. Least privilege means task-scoped credentials in a sandbox rather than standing administrative rights. An action log keeps every call and every result, so a run can be replayed and explained after the fact. Anthropic's guidance recommends checkpoints where an agent pauses for human feedback and stopping conditions such as a maximum number of iterations. The NCSC guidelines for secure AI system development, published on 27 November 2023, cover the surrounding discipline across four areas: secure design, secure development, secure deployment, and secure operation and maintenance, with logging and monitoring sitting in the last of these.

None of these five is exotic. Four of them are ordinary production engineering controls that a well-run service already has for its batch jobs. What changes is that an agent will use whatever it is given, in combinations nobody enumerated, which makes the width of the allowlist the single most consequential design decision in the system.

The Model Context Protocol is an open protocol for connecting AI applications to external data sources and tools. Its specification is built on a TypeScript schema and uses JSON-RPC 2.0 messages over stateful connections with capability negotiation at the start. It names three roles: hosts, which are the applications that initiate connections; clients, which are the connectors inside a host; and servers, which are the services that supply context and capabilities. Servers offer resources, which are context and data, prompts, which are templated messages and workflows, and tools, which are functions the model can execute. Clients may offer sampling, roots and elicitation back to the server.

The reason architects care is the arithmetic. Without a shared protocol, connecting a set of AI applications to a set of internal systems is a bespoke connector per pairing, and the count grows as the product of the two sets. With one, each system needs a single server and each application a single client, and the count grows as the sum. That is the same argument that made the Language Server Protocol worth having in developer tooling. The specification claims the parallel modestly: MCP takes some inspiration from the Language Server Protocol, which standardises how support for programming languages is added across an ecosystem of development tools, and MCP standardises the equivalent step for context and tools.

What the protocol does not settle is trust. The specification is explicit that it cannot enforce its security principles at the protocol level, and it puts the obligations on implementors: users must explicitly consent to and understand data access and operations, hosts must obtain explicit user consent before invoking any tool, and descriptions of tool behaviour such as annotations should be considered untrusted unless they come from a trusted server. That last clause carries the most weight. A tool description is text that an agent reads and acts on, which makes a server you do not control an input channel into your own decision-making.

The EU AI Act sorts systems by risk into four levels. Unacceptable risk covers practices judged a clear threat to the safety, livelihoods and rights of people, and those are banned. High risk covers systems that can pose serious risks to health, safety or fundamental rights, including applications in employment, education, critical infrastructure and law enforcement, and those carry strict obligations. Transparency risk covers situations where people should be told they are dealing with a machine, and carries disclosure duties. For minimal or no risk, which the Commission says is the vast majority of current EU applications, the Act introduces no rules.

The classification follows the use, not the model. The same base model can sit in two different levels depending on what it is pointed at, which makes tiering a deployment question rather than a procurement one, and makes the register of deployments the artefact that actually carries your compliance position. General-purpose AI models are handled separately again, with duties that fall on the model provider rather than only on whoever deploys the model.

On dates, take them from the Commission rather than from commentary, because the Act has one general date and several exceptions, and a summary tends to quote whichever it saw first. The Commission states that the Act entered into force on 1 August 2024 and will be fully applicable two years later on 2 August 2026, with some exceptions. The exceptions it lists are these: prohibited practices and AI literacy obligations entered into application from 2 February 2025, governance rules and obligations for general-purpose AI models became applicable on 2 August 2025, rules for systems used in certain high-risk areas will apply from 2 December 2027, and for systems integrated into products such as lifts or toys the rules will apply from 2 August 2028. Plan the transparency disclosure duties against the general date, because they are not among the exceptions the Commission carves out, which makes 2 August 2026 the deadline nearest to hand for most deployments.

The UK made a different bet. The 2023 white paper on AI regulation set five cross-cutting principles for existing regulators to apply: safety, security and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress. It stated plainly that the government would “not put these principles on a statutory footing initially”, and anticipated a duty on regulators to have due regard to them when parliamentary time allowed. The AI Security Institutedescribes itself as a research organisation within the Department for Science, Innovation and Technology. It evaluates and publishes, and it does not describe itself as a regulator or claim any regulatory power. Read it alongside the white paper and the position is consistent: the duties that bind a UK deployment still come from the existing regulators and from data protection law, and the institute supplies evidence into that system rather than sitting above it.

Section 80 of the Data (Use and Access) Act 2025 replaces Article 22 of the UK GDPR with new Articles 22A to 22D, and Schedule 6 carries the consequential amendments through the UK GDPR and the Data Protection Act 2018. The section came into force at Royal Assent for specified purposes, and for the remaining purposes on 5 February 2026 by commencement regulations. That is the date to check when somebody tells you the rules on automated decision-making have not changed.

The operative test is not whether a model was involved. It is whether a person was. The Act defines a decision as based solely on automated processing where there is no meaningful human involvement in the taking of it, and directs that when considering whether there is meaningful human involvement, a person must consider, among other things, the extent to which the decision is reached by means of profiling. A rubber stamp is not involvement. A caseworker who cannot see the inputs, cannot reach a different answer and has no time budget to try is closer to a stamp than to a decision-maker.

Where a qualifying automated decision is permitted, safeguards attach. They must provide the data subject with information about the decisions taken in relation to them, enable them to make representations about those decisions, enable them to obtain human intervention on the part of the controller, and enable them to contest the decision. Read as a design brief rather than as a legal footnote, that is a specification for four product features: a notification, a representations channel, an escalation route to a human with authority, and a contest path. None of them appear by accident, and none of them can be added convincingly after launch.

The lawful basis question sits underneath all of this and is often answered last, which is the wrong order. A model trained or prompted on personal data inherits the basis on which that data was collected, and a purpose the original notice did not describe is a new purpose whatever the technology.

Two anchors do most of the work, and they are not alternatives. ISO/IEC 42001:2023, first edition, published on 18 December 2023, is the management system standard for artificial intelligence. It specifies requirements and guidance for establishing, implementing, maintaining and continually improving an AI management system within an organisation, and it applies to any organisation that provides or uses products or services using AI systems, whatever its size or type. AI risk is then handled through documented policy, accountability and review rather than case by case. Third-party certification against it is now accredited in the UK: UKAS granted BSI the first accreditation for certifying AI management systems to ISO/IEC 42001:2023 on 15 January 2026. That is what changes an auditor's question and what a customer's assurance questionnaire will demand.

The NIST AI Risk Management Frameworkdoes something different. Released as version 1.0 on 26 January 2023, it is voluntary, organised around four functions, govern, map, measure and manage, and applied through profiles written for a particular setting. NIST released the Generative AI Profile on 26 July 2024 to help organisations identify the risks posed uniquely by generative AI. Because nobody can be found in breach of it, its value is as a shared vocabulary for arguing about AI risk inside an organisation, which is a more useful thing than it sounds when the argument is between a product team and a risk committee who do not otherwise share a word.

Layer them onto the accountability structure you already have rather than beside it. Under three lines of defence, the product team that builds and operates the system owns the first line, the risk or compliance function that sets the AI policy and reviews the deployment register owns the second, and internal audit tests both. Standing up a separate AI ethics board that sits outside all three is the most common way to produce a governance artefact nobody is accountable for.

AI is unusually unforgiving of the conditions it arrives into, and the reason is structural rather than technical. Conventional automation fails loudly. A batch job against a malformed file throws an exception, and somebody is paged. A generative system given the same file produces a fluent, plausible, wrong answer and nobody is paged at all. Poor data quality stops being an operational nuisance and becomes an invisible correctness problem, which is a far more expensive category.

The same asymmetry applies to process. Retrieval only helps if there is a defined corpus with an owner, a refresh cadence and a rule for what leaves it. A data contract that names the schema, the owner and the expectations is what turns a shared folder into something an AI system can be pointed at responsibly. Where an organisation has already built the layered pipeline discipline of the medallion architecture, the retrieval corpus has an obvious home. Where it has not, the corpus becomes whatever a team could get read access to, which is how a model ends up quoting a superseded policy with total confidence.

Undocumented process is the third weakness, and the one most often mistaken for a reason to adopt AI rather than a reason to wait. You cannot place a human checkpoint in a process nobody has written down, because there is no agreed step to place it at. You cannot evaluate an assistant against a task whose correct answer the organisation has never articulated. Programmes in this position usually discover it during evaluation design, at the point where somebody asks what a good answer looks like and three experts give three different ones.

Read that way, an AI pilot is a diagnostic instrument. It measures data ownership, process definition and review capacity, and it reports its findings as technical debt in the surrounding system. Organisations that treat a disappointing pilot as a model problem buy a different model and get the same result. Organisations that read the same pilot as a foundations problem fix the corpus, name the owner, write the process down, and find that the second attempt works with the model they already had.