Module E1 of 3 · Exam and certification

Course revision guide

45 min 6 stages distilled 7 figures

The whole course in one place, arranged for revision. Each stage below gives you its figure, the distinctions worth holding in your head, and the fastest routes back into the full material. Skim the figures first, then test yourself against the distinctions: anything you cannot explain from memory points at the stage to revisit.

The Data course as one arc, from literacy up to strategy

Each stage hands the next something usable, meanings you can trust, then models you can query, then pipelines you can operate, so the gate at the end tests judgement assembled across all six rather than the stage read most recently.

The Data course runs as one arc from record to strategic asset: each stage hands the next something usable, and the exam gate certifies the judgement rather than the recall. Security, quality and governance are not a stage; they run under all six.

The data course arc: from record to strategic asset Three regions. A serpentine spine of six stage cards: row one reads left to right as Foundations, Modelling and statistics, Engineering and platforms; row two reads right to left as Data and AI, Privacy and protection, Governance and strategy. Red arrows carry handover pills reading meanings you can trust, models you can query, pipelines you can operate, AI you can feed, and data you can defend. A last arrow, judgement you can certify, drops into the exam and certification gate at the bottom left, tinted red, which sends a pass route to a certificate and a resit route back to the weakest stage. A grey band under everything carries three chips: security, quality and governance. THE DATA COURSE ARC · SIX STAGES THEN ONE GATE STAGE 1Foundationsmeaning before machinesSTAGE 2Modelling andstatisticsshape then uncertaintySTAGE 3Engineering andplatformspipelines that surviveSTAGE 4Data and AIfeed the modelSTAGE 5Privacy andprotectionprotect the personSTAGE 6Governance andstrategydecide who answers meanings youcan trustmodels youcan querypipelines youcan operateAI you canfeeddata you candefendjudgement youcan certify GATEExam and certificationJudgement from all six stages is tested once.Two routes leave this gate.Certificate issuedReturn to the weakest stage passresit Security, quality and governance run under every stageSecurityQualityGovernance

Stage 1: What a value is before anyone interprets it, and the context that makes it a fact

How observation becomes information that survives the journey

Observation, representation, record and use each sit above one metadata facet, the unit, the definition, the collection method and the owner, so the band underneath is what carries meaning across the journey rather than a fifth stage at the end.

Data only becomes useful when representation, record, and decision use travel together with the metadata that preserves their meaning. ISO/IEC 11179-1:2023 frames metadata as the layer that makes that survival possible.

Data only becomes useful when meaning survives the journey Two regions. Top row: four cards Observation, Representation, Record, Use; arrows label the verbs encoded, stored, interpreted. The Use card is emphasised in red soft. Bottom band with a red left accent contains four dashed ghost cards aligned under each stage above, naming the metadata facet (Unit, Definition, Collection method, Owner) that protects that stage from meaning loss. Dashed tie lines connect each stage to its facet. OBSERVATION TO USE · FOUR STAGES TIED TO ONE METADATA LAYER 1DMBOK 2ObservationA real-world eventis seen or measured 2ISO 11179-1RepresentationValue, symbol,or category encoded 3DMBOK 2RecordStored with fields,timestamp, source 4UK GDQFUseQuestion, action,consequence on the day encoded as stored as interpreted for METADATA TRAVELS WITH EVERY STAGE · ISO/IEC 11179-1:2023 Metadata layer Description of data, registered for reliable reuse UNIT the physical quantity DEFINITION what the value means COLLECTION METHOD how it was captured OWNER who answers for it

Hold these distinctions

  • Data is a recorded observation; information is that observation plus the context that makes it a fact about something
  • SI prefixes are decimal and IEC prefixes are binary, so a mebibyte exceeds a megabyte by 4.8 per cent and a tebibyte exceeds a terabyte by about 10 per cent
  • A row layout keeps one record's fields together and suits writes; a columnar layout keeps one field's values together and suits analytical reads
  • Syntactic interoperability means two systems can parse the same bytes, semantic means they mean the same thing by them, and operational means the exchange survives a real process
  • Structural metadata says how the data is arranged, descriptive says what it is, and administrative says who is answerable and on what terms
  • Validity asks whether a value conforms to its rule and accuracy asks whether it matches the real world, so a value can be valid and wrong
  • Lawful and ethical are separate tests, and a collection with a sound legal basis can still cause harm that a harms-first review would have caught
Full stage summaryScenario practice

Stage 2: Giving data a shape, and saying how much you can conclude from it

Stage 2 route: from structure to uncertainty

Stage 2 runs in two halves: modules one to three give the facts a shape you can query and modules four to seven say how sure you are of what that shape returns, and they run in that order because shape without uncertainty produces confident nonsense.

Stage 2 runs in two halves. The first gives the facts a shape you can query; the second says how sure you are about what the shape returns. Skip either half and the other one misleads you.

Stage 2 route: from structure to uncertainty A route board of two bands. The upper grey band, Give the facts a shape, holds modules 01 Data modelling basics, 02 Relational and dimensional modelling, and 03 Master data and entity resolution. A red arrow between the bands is labelled structure you can query, then evidence you can defend. The lower band, tinted red, is Say how sure you are and holds 04 Mathematical foundations, 05 Probability and distributions, 06 Inference and experiments, and 07 Advanced analytics and machine learning. Every card carries a grey second line naming what it gives you. A red callout at the foot names what Stage 2 hands to Stage 3. STAGE 2 ROUTE · SEVEN MODULES · SHAPE FIRST, THEN UNCERTAINTY GIVE THE FACTS A SHAPEMODULES 1 TO 301Data modelling basicsentities, keys, relationships02Relational anddimensional modellingnormal forms and star schemas03Master data and entityresolutionone customer, one identifier SAY HOW SURE YOU AREMODULES 4 TO 704Mathematicalfoundationsnotation you can read05Probability anddistributionshow spread behaves06Inference andexperimentsclaims with error bars07Advanced analyticsand machine learningprediction, held honest structure you can query, then evidence you can defend What Stage 2 hands to Stage 3 A shape the engineering stage can build against, and a stated limit on what the numbers prove.

Hold these distinctions

  • A conceptual model names the things, a logical model names attributes, keys and constraints, and a physical model names the implementation
  • A natural key carries business meaning and can change; a surrogate key carries none and therefore survives
  • Third normal form removes update, insertion and deletion anomalies and costs joins; a star schema pays duplication to remove them
  • The grain is a written statement of what one fact row represents, and it decides both which measures are additive and which dimensions can join
  • Deterministic matching says a field agrees or it does not; probabilistic linkage says how much that agreement should move your belief
  • The match score is a statistical output and the threshold is a business decision, so they belong to different people
  • Correlation measures how two variables move together; causation requires ruling out confounding, reverse causation, selection and measurement change
  • A p-value is the probability of data this extreme if there were no effect, not the probability that the effect is real
  • Overfitting is high training accuracy with poor validation accuracy; leakage is a validation score that was never earned because the answer was in the features
Full stage summaryScenario practice

Stage 3: Moving data reliably, and knowing when a pipeline is lying to you

Stage 3 route: the lifecycle and its undercurrents

Stage 3 teaches one chain, not seven tools: each module sits under the lifecycle link it teaches rather than in module order, and security, orchestration and the other undercurrents run under all five links and are finished by no one module.

Stage 3 teaches one chain, not seven tools: generation, ingestion, storage, transformation, serving. Each module is placed under the link it teaches, and the undercurrents that belong to no single link run as a band beneath the whole board.

Stage 3 route: the lifecycle and its undercurrents Three regions. Across the top, five red-tinted lifecycle boxes read Generation, Ingestion, Storage, Transformation, Serving, joined by arrows labelled raw events, landed files, modelled tables and answers. Beneath them sit seven white module cards in two rows, covering streaming and change data capture, databases and storage, orchestration and DataOps, data as a product, architectures and pipelines, platforms and the lakehouse, and quality engineering and observability. Each card names the step it teaches and a grey dashed tie line rises to that box. A grey band at the foot carries six chips: security, data management, DataOps, architecture, orchestration and software engineering. THE DATA ENGINEERING LIFECYCLE · SEVEN MODULES TIED TO THEIR STEP 1Generation2Ingestion3Storage4Transformation5Servingraw eventslanded filesmodelled tablesanswers 03Streaming and changedata captureteaches: Generation04Databases andstorage systemsteaches: Storage02Orchestration andDataOpsteaches: Transformation07Data as a productteaches: Serving01Architectures andpipelinesteaches: Ingestion05Platforms and thelakehouseteaches: Storage06Quality engineeringand observabilityteaches: Transformation The undercurrents belong to no single stepSecurityData managementDataOpsArchitectureOrchestrationSoftware engineering

Hold these distinctions

  • ETL transforms before the destination and ELT transforms inside it, and the choice turns on the cost of compute and the need to keep a raw landing, not on which is newer
  • Idempotency is a property of a step, not of a pipeline: running it twice must leave what running it once left, which is what makes a backfill routine rather than risky
  • A scheduler expresses when a job starts; an orchestrator expresses what a job depends on, which version of the code produced a run, and whether the data it needed is fresh
  • Change data capture reads the database's own commit log, so it sees deletes and intermediate states that polling a modified timestamp will always miss
  • At-most-once loses records, at-least-once duplicates them, and exactly-once is an end-to-end property that requires the sink to take part rather than a setting on the producer
  • A warehouse manages storage and engine together, a lake is files you own with no guarantees, and a lakehouse is an open table format that gives lake files warehouse behaviour
  • Testing asserts what you predicted and fails when it stops holding; observability detects the change nobody wrote a rule for, which is why a fully passing test suite is not evidence of healthy data
  • A data product is a commitment to owner, contract, service level and support; the word applied to an unchanged dataset with an unchanged owner is vocabulary rather than an operating model
Full stage summaryScenario practice

Stage 4: Preparing data that a model can use, and keeping its provenance

Stage 4 route: feeding and governing AI systems

Stage 4 reads in one direction: each of the five modules supplies the hub with one nameable thing, the shapes, the index, the corpus, the trace and the definition, and the hub feeds model behaviour in production, so a weak supply returns as behaviour no prompt will fix.

Stage 4 reads in one direction: the model is downstream of the data. Each of the five modules supplies the model with one nameable thing, and if that thing is weak the model behaviour is weak with it.

Stage 4 route: feeding and governing AI systems Two regions. Along the top, five white module cards: 01 Data for AI foundations, 02 Retrieval and RAG pipelines, 03 Training data, synthetic data and provenance, 04 Catalogues, lineage and active metadata, 05 Semantic layer and metrics governance. A red arrow drops from each card into the hub below, carrying a pill naming what that module supplies: shapes the model can consume, an index over unstructured text, a defensible corpus, a trace back to the source, and a definition the agent can trust. The red-tinted hub reads The model is downstream of the data. One further arrow, labelled downstream of every choice above, leads to a card reading Model behaviour in production. STAGE 4 ROUTE · FIVE MODULES FEED ONE MODEL 01Data for AIfoundations02Retrieval and RAGpipelines03Training data,synthetic dataand provenance04Catalogues,lineage andactive metadata05Semantic layerand metricsgovernanceshapes the modelcan consumean index overunstructured texta defensiblecorpusa trace back tothe sourcea definition theagent can trust THE ONE RULEThe model is downstream of the dataEvery weakness in the corpus, the index, the lineage or definitions shows up later as model behaviour.Fix the data before you rewrite the prompt.RESULTModel behaviour in productiondownstream of every choice above

Hold these distinctions

  • A model is retrained rarely and the data under it changes continuously, so most of the variation in an AI system's behaviour over a year comes from the data rather than the model
  • An embedding preserves the neighbourhood, not the words, which is why paraphrase is found well and rare codes and surnames are found badly
  • AI-ready is a test applied to one dataset for one intended use against four components, not a property a dataset acquires permanently or a certificate anyone issues
  • Training-serving skew is a definition problem, not a modelling problem: the same feature calculated twice, in two places, by two people, at two times
  • Chunk size is settled before anything is measured and constrains every later stage, because a passage is the smallest unit retrieval can ever return
  • EU AI Act Article 10 governs the data used to build a high-risk system; Article 53 governs providers of general-purpose models, including copyright policy and a public training-content summary
  • A glossary settles what a term means and a catalogue records what a dataset is; declared metadata decays silently while active metadata is emitted by the running system
  • Column-level lineage separates a direct dependency, where the output is derived from the input, from an indirect one, where a join or filter shaped the result without the column appearing in it
Full stage summaryScenario practice

Stage 5: The lawful route, the identifiability question and defensible deletion

Stage 5 route: law defines, techniques apply, controls enforce, clocks end it

Stage 5 is one route, not four topics: definitions become a technique, the technique leaves a dataset that still needs a boundary, and the boundary still needs an end date, so protection is only complete once the clock is set.

Stage 5 is one route, not four topics: law supplies the definitions, anonymisation applies them to a dataset, security enforces who may see what, and retention puts a clock on every copy that survives.

Stage 5 route: law defines, anonymisation applies, security enforces, retention ends A four-station route board for Stage 5, read from the top down. A numbered rail runs down the left margin. Station one, Data protection law, adds the definitions of personal data, lawful basis and rights. Station two, Anonymisation and privacy-enhancing technologies, applies those definitions to a dataset. Station three, Data security essentials, enforces who may see what. Station four, Retention and deletion, puts a clock on every copy. Arrows down the rail carry the handover between stations. A red card at the foot names what the four stations together leave you holding. STAGE 5 ROUTE · FOUR MODULES THAT EACH ADD ONE THING MODULE 1 Data protection law ADDS the definitions: personal data, a lawful basis, and the rights a person can use MODULE 2 Anonymisation and PETs ADDS a technique that applies those definitions to a dataset and cuts re-identification MODULE 3 Data security essentials ADDS enforcement: who may see what, proved by access control, encryption and logging MODULE 4 Retention and deletion ADDS a clock: how long each copy may live, and the evidence that deletion happened 1 2 3 4 hands on the definitions the next module applies hands on a dataset that still needs a boundary hands on a protected dataset that needs an end date What the four stations leave you holding A lawful basis, a dataset that no longer identifies people, access that is enforced, and a deletion you can prove on the day someone asks.

Hold these distinctions

  • A lawful basis is chosen and recorded before collection and cannot be switched later; the seven UK bases have no hierarchy, so the purpose decides which one applies
  • Adequacy is checked first and the Data Privacy Framework or UK Extension only works for an actively certified, eligible recipient; everything else needs clauses plus a transfer risk assessment
  • Pseudonymised and tokenised data are still personal data because the organisation holds the means to reverse them; only genuine anonymisation leaves the regime, and that is a risk judgement about a release
  • k-anonymity is a property of one table measured against today's auxiliary data and does not compose across releases; differential privacy attaches its guarantee to a person and holds against auxiliary data nobody has seen
  • Encryption protects the stored and transmitted form and says nothing about who is asking, so access control, not encryption, is the layer that limits what a permitted reader can see
  • Masking destroys the value at the point of display and cannot be reversed; tokenisation preserves the value in a vault and preserves joins, so it relocates the secret rather than removing it
  • A retention clock starts at a business event, not a row creation timestamp, and a legal hold suspends that clock rather than resetting it
  • Defensible deletion is a claim about a designed, approved, consistently applied and recorded process, not a claim that no copy of the record survives anywhere
Full stage summaryScenario practice

Stage 6: Deciding who owns what, and treating data as an asset with a return

Stage 6 rises from decision rights to the value a board agrees to fund

Stage 6 climbs: decision rights make sharing lawful, lawful sharing makes open publication safe, and open publication is what finally makes the value of data visible to a board that funds it.

Stage 6 climbs: decision rights make sharing lawful, lawful sharing makes open publication safe, and open publication is what finally makes the value of data visible to a board that funds it.

Stage 6 rises from decision rights to the value a board agrees to fund A rising staircase of four Stage 6 module cards, read from the bottom left step upwards. Bottom step, Governance and roles: who decides and who answers. Second step, Data sharing law: what you must now share. Third step, Open data and FAIR: what you publish for anyone. Top step, Data as an asset, tinted red: what the board should fund. An arrow rises into each step carrying the question that step answers, and an entry arrow at the foot carries the first question. A red-accented card to the right of the bottom step summarises the climb. STAGE 6 ROUTE · READ IT FROM THE BOTTOM STEP UP Each step needs the one below it to be true. MODULE 1 Governance and roles Who decides and who answers MODULE 2 Data sharing law What you must now share MODULE 3 Open data and FAIR What you publish for anyone MODULE 4 Data as an asset What the board should fund who decides what? what must we share by law? what do we publish openly? what should the board fund? The route ends in a budget Decision rights, then sharing law, then open publication, then value.

Hold these distinctions

  • Governance is the allocation of decision rights and accountability; a policy that names no decision-maker for a named asset is advice rather than governance
  • The owner is accountable and decides, the steward is responsible for meaning day to day, and the custodian runs the platform and implements the controls
  • Data mesh decentralises ownership of data products but keeps governance federated and computational, so global standards stay global
  • DMBOK maps knowledge areas, ISO/IEC 38505-1 puts data use under the governing body, and ISO/IEC 42001 supplies a management system for artificial intelligence
  • The Data Act compels access and sets FRAND terms; the Data Governance Act enables re-use and regulates intermediaries, and neither displaces data protection law
  • A horizontal EU access right is scheduled by date, whereas a UK smart data scheme exists only once secondary legislation makes it for that sector
  • Open data is about permission and access for anyone, shared data is access for named parties under agreed terms, and FAIR is about reuse, so a dataset can be FAIR and closed
  • DCAT 3 describes catalogue metadata for datasets, services and distributions; ownership, support, quality rules and change policy come from local governance and data contracts
  • Direct, indirect and risk-reduction value are three different arguments, and the cost, market and income approaches are three different methods, so a valuation must name both
Full stage summaryScenario practice

Exam technique for the two papers

The Foundation paper is 40 single-answer questions in 60 minutes, closed book, and it tests precise recall. Revise the definitions, the units and prefixes, the normal forms, the six quality dimensions and the lawful bases, because those are where a near-miss answer looks right. In the sitting, read the stem twice and decide what is actually being asked before looking at the options: many questions turn on one qualifying word.

The options are written so that measuring them does not help. The correct answer is as often the short one as the long one, and no option is there to be eliminated for being obviously silly, so eliminate on meaning rather than on shape. A precise claim beats a sweeping one, and an option that would be true in a different situation is the commonest way to be wrong.

The Practitioner paper is 8 workplace scenarios in 90 minutes, open book, scored five, three, one and zero. Rank the options rather than hunting a single right answer. Across this domain the five-point answer almost always establishes what a decision requires before acting on it: it asks what the data is for, then sets the threshold, rather than profiling everything or trusting a sign-off given for a different purpose. The three-point answer is usually real work aimed slightly wrong. Use the reference panel to confirm a number, not to learn the topic inside 90 minutes.

The primary sources behind the course

Version-sensitive claims across the course anchor to primary, dated sources, and every stage summary cites the ones its teaching rests on. They are collected here so revision starts from the authority rather than from a summary of it.

When the distinctions hold from memory and the figures feel obvious, you are ready to drill the papers and then sit the mocks.

Start the exam practiceGo straight to the mocks