Data lifecycle stage summary

9 min 8 concepts 5 figures

The Data lifecycle stage follows one unit of energy data from the meter to everything it becomes: a settlement obligation, a switch, an export payment, a regulated dataset, a privacy question, a network charge and finally an open API response. It starts with the biggest change in the estate, the move to Market-wide Half-Hourly Settlement, then works through the registration spine that decides identity, the export market that gives small generators a settled position, the instruments that carry the rules, the governance that presumes data open, the rights of the person behind the meter, the money the data moves, and the platforms that publish the market to anyone.

One argument runs through the stage. The lifecycle is mid-migration, and the professional skill is knowing both worlds and which one is authoritative when. MHHS central systems went live on 22 September 2025 and the meter point migration runs to May 2027, so the legacy agent chain and the new service roles operate in parallel, the settlement run ladder still sits at its pre-cutover lengths until early July 2027, and every dataset you touch has a before and an after. Reading a claim about GB energy data now starts with the question: which side of the migration is this describing?

The sections follow the stage's teaching order, so you can read straight through to rebuild the stage in your head, or jump to the concept you need. Each section links back to its module for the full treatment.

What you carry out of this stage

  • Name the MHHS target operating model roles and say which part of the legacy MOP, DC and DA chain each one replaces
  • Trace one half-hour from the meter through the Smart Data Service, the DIP and the Market-wide Data Service into settlement, and state the run ladder before and after the M16 cutover
  • Walk a switch end to end through the Central Switching Service and name the records whose quality makes or breaks it
  • Explain export MPANs and registers, the Smart Export Guarantee's data prerequisites, and how export enters half-hourly settlement
  • Place a data obligation in the instrument hierarchy of Acts, licences and codes, and apply the presumed-open triage and the seven lawful bases with the DAPF's consent ladder
  • Read the charging data that consumes settlement outputs: TNUoS tariffs, DUoS charging statements and BSUoS
  • Choose the right open platform for a market data question across Insights, IRIS, the NESO Data Portal and the Carbon Intensity API, and read a dataset licence before reusing the data

The route through GB energy system data, from meters to certification

Each stage answers one question and hands the next what the pill under the arrow names, so the exam at stage four is asked of the vocabulary, the lifecycle and the judgement together rather than of the last stage alone.

Four stages run from vocabulary to proof: foundations, the data lifecycle, markets in motion, then exam and certification, with codes, Ofgem and data policy governing the route from above. Source: this course's syllabus, 31 modules across four stages.

The route through GB energy system data, from meters to certification Four stage nodes run left to right inside a thin frame: stage 1 Foundations, 6 modules, asking what the system and its data are; stage 2 The data lifecycle, 8 modules, asking how a reading becomes money; stage 3 Markets in motion and strategy, 8 modules, asking where the market is going; stage 4 Exam and certification, 3 exam surfaces, asking whether you can prove it. Labelled arrows between the stages read builds the vocabulary, follows the data and applies the judgement. A governance band above stages 1 to 3, labelled codes, Ofgem and data policy, drops into stage 2 (rules) and stage 3 (strategy). The The data lifecycle stage is highlighted as the current stage. CODES · OFGEM · DATA POLICY stages 1 to 3 rules live in stage 2 strategy lives in stage 3 STAGE 1 Foundations 6 modules What is the systemand its data? STAGE 2 The data lifecycle 8 modules How does a readingbecome money? STAGE 3 Markets in motionand strategy 8 modules Where is the marketgoing? STAGE 4 Exam andcertification 3 exam surfaces Can you prove it? builds the vocabulary follows the data applies the judgement

Settlement runs on MHHS now, and the run ladder compresses at M16

Market-wide Half-Hourly Settlement is the present tense. Central systems went live on 22 September 2025, the 18-month meter point migration began on 22 October 2025, roughly 80 percent of meters are expected in the new arrangements by October 2026, and migration completes in May 2027. The target operating model replaces the legacy MOP, DC and DA agent chain with service roles: the Smart Data Service and Advanced Data Service retrieve readings, the Metering Service Smart and Metering Service Advanced look after the assets, the Load Shaping Service replaces profiling by supplying a shaped estimate only where an actual half-hourly read is missing, and the Market-wide Data Service absorbs aggregation. All of it is wired through the DIP, the Azure-based message router live since August 2025 with Avanade as its service provider.

The trace discipline still anchors the module: 1.47 kWh recorded in one half-hour travels from the meter to the Smart Data Service, through the DIP to the Market-wide Data Service, and into central settlement. Two corrections keep the chain honest. The SVAA is one central Elexon service, with volume allocation and GSP Group correction operating across the 14 GSP Groups, and the GB imbalance price is a single national price, with regional variation entering only through loss factors and network charges.

Timing is a fact you can be wrong about. Initial Settlement, the SF run, lands 16 working days after the settlement day, and the SF volume allocation run at 15 working days is the classic source of confusion. After the M16 settlement timetable cutover in early July 2027, the SF-equivalent run drops to about 7 working days and final settlement to about 4 months. Around the chain sit three platforms with three fates: the legacy DTS transfer network is being wound down, the DIP is live, and the DSI is a sharing layer still being coordinated by NESO to 2028.

Seven stages take a half-hour reading from the meter to a bill

You can only name the rulebook and the owner for a missing reading once you have established where it stopped: the owner changes at almost every stage, and the rulebook pill changes five times across the seven.

Seven stages, six rulebooks, one reading, with the legacy agents in the outgoing lane. Source: BSC Sections S and T, SEC Schedule of Services, BSCP701, MHHS Programme, supplier licence.

Seven stages take a half-hour reading from meter to bill under MHHS Seven cards in a horizontal chain: Capture, Store, Transmit, Validate, Aggregate, Settle, Bill. Each card states the stage number, name, owner, a rulebook pill (from SEC, its sections, the BSCPs and BSC sections, or the supplier licence) and the question that stage answers. Owners carry the MHHS role names: the Smart Data Service validates and the Market-wide Data Service aggregates, fed by the DIP. Validate and Bill are emphasised as the two stages most likely to fail or surprise consumers. A dashed lane beneath shows the legacy agents retiring by May 2027: the MOP becomes the Metering Service, the Data Collector the Smart Data Service, and the Data Aggregator the Market-wide Data Service. 01 Capture OWNER Meter RULEBOOK SEC What kWh in 30 min? 02 Store OWNER Meter buffer RULEBOOK SEC Reading saved? 03 Transmit OWNER DCC WAN RULEBOOK SEC §H Read reached DCC? 04 Validate OWNER SDS RULEBOOK BSCP701 Read plausible? 05 Aggregate OWNER MDS via DIP RULEBOOK BSC §S Supplier volume? 06 Settle OWNER Elexon SAA RULEBOOK BSC §T Who owes how much? 07 Bill OWNER Supplier RULEBOOK Licence Consumer pays what? OUTGOING LANE · LEGACY AGENTS RETIRE BY MAY 2027 MOP becomes Metering Service DC becomes Smart Data Service DA becomes Market-wide Data Service
Go deeper in Module 1: From meter to market: settlement under MHHS

Switching is a data transaction on the CSS, and address quality is the binding constraint

The Central Switching Service has been the registration source of truth for both fuels since July 2022. It is operated by the DCC and governed under the Retail Energy Code, with the MPXN and the Retail Energy Location address record at its core. Consolidating registration is what turned a switch from a paper chase into a data transaction: an enquiry through the Gas and Electricity Enquiry Services, a CSS registration update, service appointments on the electricity side under the MHHS roles, industry notifications, and a first bill, with a named dataset touched at every step.

Quality, not speed, is now the constraint. Ofgem decided in December 2024 to keep the Centralised Registration Service with the DCC, and the joint RECCo and DCC improvement plan for 2025-26 exists because address data problems still fail switches: when the Retail Energy Location record and the billing address disagree, the mismatch surfaces as a failed or misdirected switch, and the cost lands market-wide.

The DCC2 transition is the stage's forward risk. The DCC's current licence expires in September 2027, Ofgem's successor licence decision came in April 2026, and the business transfer is expected in November 2026, which makes the handover a registration and WAN data continuity question rather than a distant procurement story. A learner who can walk a switch and name where it breaks can also say precisely what must not be dropped in that handover.

Go deeper in Module 2: Switching and registration: the CSS data spine

Export is not negative import: separate register, separate MPAN, and increasingly a settled market

The physics of the meter is the starting fact: exported electricity is recorded on its own export register against a separate export MPAN, and the import register never goes negative. A solar home is therefore two meter points, not one meter running backwards, and everything commercial about microgeneration follows from that separation.

The Smart Export Guarantee made export data commercially load-bearing. Suppliers with 150,000 or more domestic customers must offer export tariffs, and payment requires half-hourly export metering, so the export register is not a curiosity but the evidence a payment obligation rests on. Under MHHS, export MPANs enter half-hourly settlement in their own right, which changes the position of storage and co-located assets, where the metering arrangements decide how a single site's flows are split and settled.

Where meters end, models begin. GB's embedded solar is largely invisible to the system operator, so PV_Live, run by Sheffield Solar with NESO, estimates GSP-level generation from a sample of monitored sites. The teaching point is that an estimate published with error bars is still a data product, consumed operationally every day, and the stage treats it with the same seriousness as a metered series.

Go deeper in Module 3: Export data: microgeneration, SEG and storage

The rulebook stacks Acts, licences and codes, and it grows a new instrument roughly annually

Data obligations arrive through a hierarchy. Acts of Parliament, from the Electricity Act 1989 through the Energy Act 2023 to the Data (Use and Access) Act 2025, set the frame; licence conditions make duties enforceable, with the distribution condition that requires compliance with the Data Best Practice Guidance and a published Digitalisation Strategy and Action Plan turning data governance into a regulated obligation; and the seven industry codes (BSC, REC, SEC, Grid Code, DCUSA, CUSC and UNC) carry the operational detail of who must send what to whom.

The RIIO price controls fund digitalisation on five-year cycles, and code modification typically takes 12 to 18 months. That pacing is the innovation-against-regulatory-pace tension made institutional: the data infrastructure risks running one regulatory cycle behind the technology, which is why the Energy Act 2023's code governance powers, pointing towards consolidation, matter for data people and not just lawyers.

The rulebook is also in motion. The Smart Secure Electricity Systems programme is adding a new layer for energy smart appliances, with the load control licensing consultation running December 2025 to February 2026, and the Data (Use and Access) Act 2025 reshaped the Smart Data landscape the energy sector will build on. The habit to take away: when you meet a data obligation, place it on the ladder first, because the instrument tells you who can change it and how fast.

Five Acts of Parliament set the legal frame for GB energy data

Each row ends at a specific power rather than a principle, so a GB energy data obligation traces back to one of these five Acts, and the two emphasised rows are where the licence and code machinery itself comes from.

Five Acts of Parliament create the legal hooks for every GB energy data obligation. Source: legislation.gov.uk; Ofgem section 11 register.

Five Acts of Parliament set the legal frame for GB energy data Five rows in a vertical stack, one per Act. Each row shows the year, the Act name, the key section that matters for data, and the specific power the Act grants for data obligations. Electricity Act 1989 and Energy Act 2023 are emphasised because they are the central data-power Acts: section 11 modifications and the NESO and code-reform provisions respectively. 1989 Electricity Act 1989 KEY SECTION §6 licences; §11 modifications POWER FOR DATA Licences, codes, Ofgem's modification power 1986 Gas Act 1986 KEY SECTION Pt 2 licences; UNC framework POWER FOR DATA Transporter, shipper and supplier licences 2023 Energy Act 2023 KEY SECTION Part 5 (NESO); Part 8 (code reform) POWER FOR DATA Creates NESO; data-gathering powers; code reform 2025 Data (Use and Access) Act KEY SECTION Part 1 Smart Data; Part 5 GDPR amendments POWER FOR DATA Smart Data API powers; legitimate interests 2018 Data Protection Act 2018 KEY SECTION Schedule 1; ICO supervision POWER FOR DATA Implements UK GDPR; ICO supervises personal data
Go deeper in Module 4: The rulebook: legislation, codes, and RIIO

Governance presumes data open, and sharing is getting shared infrastructure

Ofgem's Data Best Practice Guidance, at version 3.5 since 30 June 2025, presumes energy system data open. The Data Triage Playbook is the working procedure that honours the presumption: a stepwise classification that sorts a dataset towards open, shared or restricted outcomes, with openness as the default exit unless a specific harm is argued. Around it sit the FAIR principles and metadata standards that make published data findable and usable rather than merely present, and the maturity gap between the guidance and network reality is taught as a finding, not an accusation.

The sharing layer is becoming institutional. The Data Sharing Infrastructure got an owner with Ofgem's governance decision in March 2025, with NESO coordinating delivery to 2028 and early capability piloted inside the Virtual Energy System. The DSI is not a data warehouse: it is a federated layer of common governance, processes and technology that makes distributed datasets discoverable and exchangeable while they stay with their owners, and it deliberately has no fixed launch date. Alongside it, the Elexon Smart Data Repository launches in autumn 2026 from the DESNZ and Ofgem scoping work on wider access to smart metering data.

The direction of travel is the point. GB is moving from publish-your-own-portal to shared infrastructure, and control moves with it: the questions worth asking of any sharing initiative are who governs access, who pays, and what happens to the presumption of openness when datasets that are individually safe become sensitive in combination.

Three exit branches, one default: the GB energy data triage

Restriction is reached only by answering yes to one of the three questions, and every no path runs down to the open band, so the burden falls on whoever wants to withhold a dataset rather than on whoever asks for it.

Personal, commercial, security: three exit branches. Everything else defaults to open. Source: Ofgem Data Best Practice Guidance 2026.

Three exit branches, one default: the GB energy data triage A four-step decision tree drawn vertically. Three question rows ask in turn: is it personal data, is it commercially sensitive, is it security sensitive. A yes answer on any row exits to the right into a brand-red outcome card (personal route, commercial route, security route). A no answer continues downward to the next question. Any dataset that survives all three tests lands in a brand-red default-open band at the bottom, publishing under the Data Best Practice Guidance presumption. Q1 Is it personal data under UK GDPR? If yes, exit right; if no, continue down YES · EXIT Personal route Lawful basis and subject rights YES Q2 Is it commercially sensitive? If yes, exit right; if no, continue down YES · EXIT Commercial route Restricted with audited access YES Q3 Is it security sensitive (CNI risk)? If yes, exit right; if no, continue down YES · EXIT Security route Need-to-know, NCSC OT controls YES NO NO NO DEFAULT OUTCOME Open by default Publish under DBP presumption
Go deeper in Module 5: Who controls the data: governance and triage

Half-hourly data is personal, seven lawful bases apply, and consent is becoming infrastructure

Half-hourly consumption data is personal data when it can be linked to an identifiable household, because it can reveal routines and occupancy. That single fact pulls the whole UK GDPR apparatus into energy: the principles, including data minimisation with its uncomfortable question of whether half-hourly grain is necessary for each purpose, the data subject rights, and the requirement that every act of processing rest on a lawful basis.

The lawful bases now number seven. UK GDPR's six were joined by recognised legitimate interests when the Data (Use and Access) Act 2025 received Royal Assent on 19 June 2025, and in energy practice four bases do most of the everyday work: consent, contract, legal obligation and legitimate interests, each with a known cost, from withdrawable consent to the documented balancing test. Above the general law sits the sector's own rulebook: the smart meter Data Access and Privacy Framework sets the consent ladder for who may access half-hourly data and at what granularity, and PKI-E with the NIS regime secures the estate the data crosses.

Consent itself is getting infrastructure. The RECCo Consumer Consent Solution moves consent from per-supplier forms to a shared trust framework: design consultation ran February to March 2026, Raidiam and PayPoint were appointed in May 2026, an MVP is due in 2026 and the minimum marketable product is targeted for March 2027, with consumer tariff data sharing to follow in November 2027. The exam-grade habit is matching each processing purpose to its basis and its access route, rather than reaching for consent as the answer to everything.

Four UK GDPR Article 6 lawful bases for processing energy data

Data can only be used within the limit paired with its basis, so the choice made at collection fixes what can be done later: consent can be withdrawn, contract reaches only the supply purpose, legal obligation only the mandated scope, legitimate interests can be objected to.

Four lawful bases. The right choice limits what can be done later. Source: UK GDPR Article 6, Data Protection Act 2018, ICO Energy Guidance.

Four UK GDPR Article 6 lawful bases for processing energy data A two-by-two grid of four lawful basis cards labelled A, B, C and F (Article 6 paragraphs). Each card has a brand-red letter block on the left, then the basis name, the conditions under which it applies, the risk on withdrawal or challenge, and a worked example from energy data. Contract and legal obligation are emphasised because they bear the bulk of regulated processing. A Art 6(1) Consent WHEN IT APPLIES Subject chose, opt-in, recorded RISK Withdrawable; loss of processing right EXAMPLE Sharing half-hourly data with a third-party tariff agent B Art 6(1) Contract WHEN IT APPLIES Necessary to fulfil the supply contract RISK Only covers the supply purpose itself EXAMPLE Reading the cumulative register to issue a bill C Art 6(1) Legal obligation WHEN IT APPLIES Specific legislation mandates the processing RISK Limited to the mandated scope EXAMPLE BSC settlement collection of half-hourly volumes F Art 6(1) Legitimate interests WHEN IT APPLIES Documented balancing test passes RISK Subject can object; ICO can challenge EXAMPLE Network planning aggregates that benefit the wider system
Go deeper in Module 6: Your data, your rights: privacy and consent

Three BSC agents move the money, one national price clears it, and charging consumes the outputs

The money side of settlement runs through three BSC agents: the SVAA allocates supplier volumes, the SAA administers settlement and computes the imbalance positions, and the FAA administers the funds that change hands. The imbalance price they work to is a single national price, and the consumer bill decomposes into six cost components, so a learner can trace any pence-per-kWh claim to the component it belongs in rather than treating the bill as one number.

Network charging is settlement data's biggest downstream customer. TNUoS tariffs are published by NESO by 31 January each year and take effect on 1 April, so parties know their charges before the charging year opens. DUoS sits in each DNO's charging statement under the DCUSA, with the charging bands in Schedule 32, and BSUoS recovers the cost of the balancing actions the system operator took. All three are public data products: the statements and tariff tables are open, readable and dated, which makes charging one of the best places in the sector to practise reading real data artefacts.

The stage's correction discipline applies here too. Initial Settlement at 16 working days, a single national imbalance price, and the MHHS compression of the run ladder are the three facts this module holds steady, because charging and reconciliation arithmetic downstream inherit any error made about the settlement timetable upstream.

SVAA, SAA, FAA: the three BSC settlement agents

Each agent answers to its own BSC section, but its output is the next agent's input, so a volume the SVAA allocates wrongly becomes a charge the SAA raises wrongly and cash the FAA has already moved, which is what the reconciliation runs exist to correct.

SVAA, SAA, FAA: three peer agents, one chain. Source: BSC Section S; BSC Section T; Elexon Trading Operations Manual.

SVAA, SAA, FAA: the three BSC settlement agents Three peer column cards labelled SVAA (Supplier Volume Allocation Agent), SAA (Settlement Administration Agent, emphasised) and FAA (Funds Administration Agent). Each card states the full name, role, input, output and BSC section reference. A red strip below the cards labels the settlement chain SVAA feeds SAA feeds FAA and names the reconciliation run pattern. SVAA BSC §S Supplier Volume Allocation Agent ROLE Allocate metered volumes to suppliers per GSP INPUT Validated half-hour reads OUTPUT Supplier metered volumes SAA BSC §T Settlement Administration Agent ROLE Calculate imbalance and charges per Trading Party INPUT Volumes, PN, accepted B/O OUTPUT Settlement charges FAA BSC §T Funds Administration Agent ROLE Move money between Trading Parties INPUT Settlement charges OUTPUT Daily cash flows, accounts SETTLEMENT CHAIN · SVAA FEEDS SAA FEEDS FAA Reconciliation: SF, R1, R2, R3, DF
Go deeper in Module 7: Show me the money: settlement, markets and network charges

The open platforms differ by cadence and contract, not just by owner

GB market data is genuinely open, and the platforms sort by how they deliver and on what terms. The Elexon Insights Solution at bmrs.elexon.co.uk serves open RESTful APIs with no key required, and it replaced legacy BMRS, which was switched off on 31 May 2024, so any current workflow that still points at BMRS points at a retired platform. IRIS is the free push service beside it, with open-source clients, for consumers who want messages delivered rather than polled.

Two more platforms complete the map. The NESO Data Portal publishes datasets under per-dataset licences, mostly CC BY 4.0, which is why licence reading is taught as a first-class skill rather than a formality. The Carbon Intensity API at carbonintensity.org.uk, operated by NESO with Oxford, EDF Europe and WWF, publishes forecasts more than 96 hours ahead across 14 DNO-boundary regions, and the open Octopus API's Agile half-hourly unit rates let a learner cost the same appliance run at three different times of day.

The module's walkthroughs make the landscape concrete: yesterday's system price and generation mix from Insights, a Data Portal treasure hunt that ends at the licence field, the greenest half-hour from the regional carbon forecast, and a wash cycle costed on Agile rates. Behind the publishing sits REMIT, the transparency regime that obliges market participants to disclose inside information, and beside the metered series sit estimates such as PV_Live, so the closing judgement is choosing the right platform, cadence and contract for the question in hand.

Go deeper in Module 8: Hands on with market data: Insights, IRIS and the open APIs

The traps this stage warns against

  • Describing the SVAA as a service operating at each grid supply point, with a regional settlement price.

    Instead: The SVAA is one central Elexon service, volume allocation and GSP Group correction work across the 14 GSP Groups, and the GB imbalance price is a single national price. Regional variation enters through loss factors and network charges, not the settlement price.

  • Stating the Initial Settlement run at 15 working days.

    Instead: Initial Settlement (SF) lands 16 working days after the settlement day; the SF volume allocation run at 15 working days is the source of the confusion. After the M16 cutover in early July 2027 the equivalent run drops to about 7 working days.

  • Sending someone to BMRS for current market data.

    Instead: Legacy BMRS was switched off on 31 May 2024. The Elexon Insights Solution and the IRIS push service replaced it, with open REST APIs and no key required.

  • Modelling microgeneration as negative demand on the import MPAN.

    Instead: Import and export are separate registers on separate MPANs, and the import register never goes negative. Export settles in its own right under MHHS, and SEG payment depends on half-hourly export data existing.

  • Treating the DSI as a central data warehouse with a launch date to wait for.

    Instead: The DSI is a federated discovery-and-exchange layer, coordinated by NESO to 2028 and piloted under the Virtual Energy System. Datasets stay with their owners, and there is no fixed launch date.

Core distinctions

  • The Smart Data Service retrieves half-hourly readings from smart meters and the Advanced Data Service does the same job for advanced meters, so the split is by meter category, not by function
  • The Metering Service Smart and Metering Service Advanced look after the meter assets; the data services retrieve the readings, so asset care and data retrieval are separate MHHS roles
  • The DTS is the legacy transfer network being wound down, the DIP is the live Azure-based message router at the centre of MHHS, and the DSI is a data sharing layer still in coordinated delivery to 2028
  • Import and export settle as separate meter points with separate registers; export is a settled position in its own right under MHHS, never a negative import number
  • TNUoS recovers transmission network costs on tariffs NESO publishes by 31 January, DUoS sits in each DNO's charging statement under the DCUSA, and BSUoS recovers the cost of balancing actions
  • Insights is REST pull, where you request data when you want it; IRIS is push, where messages are delivered to your subscription as they publish, and the choice is cadence and architecture, not content
  • The DCC operates the Central Switching Service; the Retail Energy Code and RECCo govern it, so the operator and the governance answer to different questions

That is the Data lifecycle stage in one place. Settlement re-platformed under MHHS with a run ladder that compresses at M16, switching decided by registration records on the CSS, export as a market with its own registers and settlement position, the instrument hierarchy that carries every data obligation, presumed-open governance growing shared infrastructure, seven lawful bases and a consent framework under construction, the agents and charges that move the money, and the open platforms that publish the market. The data lifecycle scenario practice now puts the whole chain under pressure, with a capstone that follows one kWh through the Smart Data Service, the DIP and the Market-wide Data Service into imbalance settlement and diffs it against the legacy chain, before stage 3 turns to the markets and strategy being built on top.

Start the scenario practice

Sources and further reading