MODULE 2 OF 5 · PRACTICE AND STRATEGY

The Digital Future: Seven Transformation Programmes

35 min read 3 outcomes Practice & Strategy

By the end of this module you will be able to:

  • Describe DIP, MHHS, CCS, DSI, FMAR, Smart Data, and Clean Power 2030 with their current status
  • Explain how AI and digital twins are transforming grid planning
  • Assess the 2030 vision for GB energy data infrastructure

12.1 The seven programmes mapped

Seven transformation programmes reshape GB energy data to 2030

Four-column grid of seven programmes (DIP, MHHS, CCS, DSI, FMAR, Smart Data, CP30) with status chip, milestone date, outcome and lead body.

Seven transformation programmes reshape GB energy data to 2030 A four-column grid of seven programme cards (DIP, MHHS, CCS, DSI, FMAR, Smart Data, CP30). Each card shows the abbreviation, the full programme name, a status chip (LIVE / IN BUILD / IN DESIGN), the milestone date, the outcome and the lead body. DIP and Clean Power 2030 are emphasised because they bracket the timeline: DIP went live first; Clean Power 2030 is the destination. DIP Data Integration Platform LIVE Live Aug 2025 Real-time settlement messaging LEAD · ELEXON MHHS Market-wide Half-Hourly IN BUILD M15 May, M16 Jul 2027 Half-hour settlement LEAD · ELEXON, SUPPLIERS CCS Consumer Consent Service IN BUILD In build 2025-26 Standardised consent capture LEAD · RECCO DSI Data Sharing Infrastructure IN DESIGN Interim to 2028 Cross-platform discovery LEAD · NESO, OFGEM FMAR Flexibility Market Arrangements IN BUILD Phased delivery Flexibility market rules LEAD · ELEXON Smart Data Energy Smart Data Scheme IN DESIGN Scheme design Customer data access LEAD · DESNZ, OFGEM CP30 Clean Power 2030 Plan IN BUILD Plan published 2024 Clean power by 2030 LEAD · DESNZ built by ransfordsnotes.com

Seven programmes, three statuses, one decade. Source: MHHS, Elexon DIP, NESO DSI, RECCo CCS, Elexon FMAR, DESNZ Clean Power 2030, DUA Act 2025.

DIP: Data Integration Platform

Elexon's Data Integration Platform is the live market data exchange platform built for Market-wide Half-Hourly Settlement and related electricity market messages. It is not a NESO planning database and it is not the Data Sharing Infrastructure. Its job is narrower and more operational: move electricity market participant data using modern event-driven services so MHHS can run at scale.

Elexon reported in March 2026 that the DIP had processed more than one billion messages since August 2025. That matters because it proves the new settlement messaging layer is already live while other digitalisation programmes are still in design or staged delivery.

DSI: Data Sharing Infrastructure

The Data Sharing Infrastructure is led by NESO and represents the most fundamental change to how energy data is discovered and accessed. The DSI is not a data warehouse, it does not store energy data. Instead, it functions as a discovery and trust layer: a catalogue of what data exists, who holds it, under what terms it can be accessed, and what consent is required. Think of it as a search engine for energy data, combined with an identity and consent management system.

Ofgem appointed NESO as interim DSI coordinator for 2024 to 2028. The current status is staged development rather than a single fixed launch date. Early work focuses on common governance, catalogues, trust services, access processes and technical patterns that allow data to remain with multiple custodians while still being discoverable and shareable.

The DSI matters because it addresses the fundamental problem identified in Module 11: GB's energy data is fragmented across multiple platforms with no unified way to discover what exists or request access. Without the DSI, every new data- driven innovation requires bespoke bilateral agreements with each data holder. With the DSI, innovators can discover available data, understand access terms, and manage consent through a single interface.

FMAR: Flexibility Market Arrangements Review

The Flexibility Market Arrangements Review is Elexon's programme for improving how flexibility market arrangements work across the GB electricity system. It is closely connected to asset visibility, dispatch, settlement and interoperability, but it should not be reduced to a single asset-register product unless a current source uses that narrower term.

Currently, NESO has reasonable visibility of large-scale generation and transmission- connected assets, but limited awareness of the rapidly growing fleet of distributed resources connected to the distribution network. A 50 MW battery farm connected at 132 kV is well understood. Ten thousand domestic batteries aggregated by a virtual power plant operator are largely invisible to system operation. FMAR addresses this gap by creating a standardised register of flexibility assets, their technical capabilities, and their availability for dispatch.

Smart Data: energy smart data scheme

The energy smart data scheme is the policy route for secure, customer-authorised sharing of energy data with authorised third parties. It is enabled by the Data (Use and Access) Act 2025 and subsequent scheme design work. The current source-backed framing is an energy smart data scheme, not a final central repository with a settled operating model.

The Open Banking parallel is instructive. When Open Banking launched in the UK in 2018, it faced scepticism about whether consumers would actually use it. Eight years later, millions of consumers use Open Banking-powered services for budgeting, switching, and credit assessment. Energy Smart Data aims to create a similar consented access layer for energy, enabling consumer-facing services built on standardised data access rather than proprietary integrations.

CCS: Consumer Consent Service

RECCo is developing the Consumer Consent Service as the consent management framework for customer-directed energy data sharing. The CCS provides a standardised mechanism for consumers to grant, revoke, and manage consent for their energy data to be shared with third parties. It addresses the current problem where consent is managed inconsistently across different platforms, making it difficult for consumers to understand who has access to their data and how to withdraw that access.

The consent framework is not a trivial problem. Energy data consent must handle granular permissions (which data types, for what purpose, for how long), delegated consent (a landlord consenting on behalf of tenants), and revocation that propagates across all downstream systems. The CCS is designed to handle all of these scenarios through a centralised service that market participants integrate with, rather than implementing their own consent mechanisms.

MHHS: Market-wide Half-Hourly Settlement

MHHS is the most operationally complex of the seven programmes. Official programme milestones distinguish full transition from the shorter settlement timetable: M15 is 7 May 2027 and M16 is 2 July 2027. This is a step change from the current system where many domestic and small-business meter points are settled using static profiles that estimate consumption patterns rather than actual half-hourly readings.

The data scale is staggering: MHHS will generate approximately 500 billion meter readings per year. That estimate should be cited directly from Elexon rather than derived from a broad all-meter count, because the settlement denominator is not the same thing as the installed smart and advanced meter estate. The infrastructure required to collect, validate, aggregate, and settle this volume is fundamentally different from the current batch-processing approach.

MHHS matters for settlement accuracy and for creating the price signals that drive demand-side flexibility. Under profile settlement, a domestic consumer who shifts their washing machine and EV charging to off-peak hours receives no financial benefit because the profile assumes average behaviour. Half-hourly settlement rewards actual demand shifting, creating the economic signals needed for a flexible, decarbonised system. The transition from estimated to actual settlement is one of the most consequential data infrastructure changes in GB energy history.

CP30: Clean Power 2030

Clean Power 2030 is the policy destination that gives the data programmes urgency. It is not a data platform, but it changes the performance requirement for every data platform. A system with more renewables, more flexibility, faster connections and more active demand cannot be planned or operated using slow, incomplete, inconsistent data.

Communications modernisation, including VWAN for hard-to-reach premises, remains a supporting dependency for smart metering reliability. It is not treated here as one of the seven headline transformation programmes because the programme map is focused on sector data, markets, consent and planning outcomes to 2030.

Check your understanding

What is the primary function of the DSI (Data Sharing Infrastructure)?

MHHS will generate approximately 500 billion meter readings per year. The data infrastructure required for this volume is fundamentally different from batch processing.

Elexon, Market-wide Half Hourly Settlement and the BSC

This scale - half a trillion readings per year - transforms settlement from a batch process into a continuous data management challenge. The shift from estimated profiles to actual readings requires infrastructure capable of near-real-time validation and aggregation at a scale unprecedented in GB energy markets.

The seven transformation programmes create the data infrastructure. Section 12.2 examines the AI and digital twin applications that will run on top of that infrastructure once it is in place.

12.2 AI and digital twins in grid planning

Beyond the seven formal programmes, two technology trends are transforming how the energy system uses data: artificial intelligence and digital twins. Neither is a formal programme with a governance board and a delivery timeline. Both are being adopted organically by system operators, network companies, and market participants as the data foundations improve.

AI and digital twins sit on data foundations: five-layer dependency stack

Vertical stack from operational decision at the top down through digital twin, AI model, feature store, and raw observation. A 'depends on' axis arrow rises on the left.

AI and digital twins sit on data foundations: five-layer dependency stack A vertical stack of five layers. The top layer is the operational decision (reinforce, defer, flex). Below it sits the digital twin running counterfactuals. Below that the AI or analytics model. Below that the feature store with cleaned and joined data. The bottom layer is raw observation data (half-hour reads, SCADA, assets, weather). A brand-red 'depends on' arrow runs up the left axis showing that the higher capabilities cannot exist without the lower layers. Layer 5 decision and Layer 1 raw data are emphasised as the input and the output of the stack. DEPENDS ON L5 Operational decision Network operator dispatches reinforcement, defer or flex Example: Reinforce feeder X by 2027 L4 Digital twin Synchronised simulated network used to test counterfactuals Example: What-if cold snap with 10% more heat pumps? L3 AI / analytics model Trained model predicts load, fault, capacity headroom Example: LV load forecast at every secondary substation L2 Feature store Cleaned, joined data with lineage; CIM-aligned where possible Example: Half-hour load times weather times demographics L1 Raw observation Half-hour reads, SCADA telemetry, asset records, weather Example: Up to 500bn MHHS readings per year built by ransfordsnotes.com

AI sits on data; digital twins sit on AI. Five layers from raw observation to operational decision. Source: NESO Virtual Energy System; National Grid Digital Twin papers.

Demand forecasting and carbon intensity

AI-powered demand forecasting is already operational in GB. NESO uses machine learning models to forecast national demand, wind output, and solar generation. The National Grid ESO Carbon Intensity API provides 30-minute granularity carbon intensity data across 14 GB regions, with 96-hour forward forecasts. These forecasts use neural networks trained on historical generation mix, weather data, and interconnector flows to predict the carbon intensity of electricity at each half-hour interval.

The practical impact is already significant. A data centre operator using the Carbon Intensity API can shift computational workloads to periods of low carbon intensity, reducing both emissions and costs. An EV charging network can schedule overnight charging to coincide with high wind output. A smart heating system can pre-heat a building when carbon intensity is low and coast through high-carbon periods. These applications already exist and are growing rapidly as the API becomes more widely adopted across industries.

Predictive maintenance

Network companies are deploying machine learning for predictive maintenance of distribution assets. By analysing patterns in transformer loading, cable temperature, partial discharge measurements, and historical fault data, these models can predict equipment failures before they occur. The value is not just in avoided outages and improved customer service but in optimised capital expenditure: replacing assets based on condition rather than age reduces the total cost of maintaining the distribution network.

The data dependency is crucial. Predictive maintenance models are only as good as the sensor data and operational history they are trained on. As CIM standardisation improves the quality and consistency of network model data across all 14 DNOs, and as smart meter data provides higher-resolution demand information at each connection point, these models become progressively more accurate. The virtuous cycle between better data and better AI is already visible in early deployments.

Digital twins

A digital twin of the electricity network is a virtual model that mirrors the physical network in near-real-time. It ingests live data from SCADA systems, smart meters, weather stations, and market systems to create a dynamic representation of the network state. Planners can then simulate scenarios, connecting a new wind farm, closing a substation for maintenance, managing a cluster of heat pumps in a residential area, without affecting the real network.

NESO and several DNOs are developing digital twin capabilities. The LTDS CIM models (described in Module 14) provide the structural foundation: a standardised representation of every network element, its connectivity, and its electrical parameters. The digital twin adds live operational data on top of this structural model, creating a dynamic representation that updates continuously as conditions change.

The 2030 vision is a whole-system digital twin of the entire GB electricity system from 400 kV transmission to low-voltage distribution. This would enable integrated planning across voltage levels, real-time congestion management, and automated response to system events. Whether this vision is achievable by 2030 depends entirely on the quality and availability of the underlying data, which is precisely what the seven transformation programmes are designed to deliver.

AI and digital twins are already emerging where data quality is sufficient. Section 12.3 assesses whether the 2030 vision - all seven programmes operational, all dependencies resolved - is achievable on the current timelines.

12.3 The 2030 vision assessed

The 2030 vision assessed: five outcomes with readiness chips

Five outcome rows scored ON TRACK, AT RISK or OFF TRACK with the dependency programme that drives each.

The 2030 vision assessed: five outcomes with readiness chips A five-row stack of outcome cards. Each outcome states the name (half-hour settlement, consent-bound flows, cross-platform discovery, interoperable network model, Clean Power 2030 evidence), a one-line description, a brand-red ON TRACK chip, a white AT RISK chip or a neutral OFF TRACK chip, and the dependency programme that drives it. OUTCOME 01 Half-hour settlement for all All meter points settled HH ON TRACK MHHS Jul 2027 OUTCOME 02 Consent-bound consumer data flows CCS-mediated third-party access AT RISK CCS in build OUTCOME 03 Cross-platform discovery Catalogue across DCC, DIP, DTS, etc. OFF TRACK DSI in design OUTCOME 04 Interoperable network model CIM-aligned topology + capacity ON TRACK LTDS Stage 3 OUTCOME 05 Clean Power 2030 evidence Connections queue cleared; flex live AT RISK CP30 Plan + FMAR built by ransfordsnotes.com

Five outcomes; three on track, two at risk. Source: DESNZ Clean Power 2030 Plan; NESO FES 2024; Ofgem strategy papers.

The converging timelines of all seven programmes create a vision of what GB energy data infrastructure could look like by 2030. In this vision, all meter points are settled half-hourly after the MHHS full transition and shorter settlement timetable milestones. All flexibility assets above a minimum threshold are registered and visible to system operators through improved flexibility market arrangements. Consumers can access and share their energy data through standardised APIs under an energy Smart Data scheme. Consent is managed transparently through the Consumer Consent Service. Data discovery and access are improved through a DSI trust and catalogue layer. Communications infrastructure supports more reliable data flows, including hard-to-reach premises.

The critical question is interdependency. Smart Data needs the Consumer Consent Service for consent management. FMAR needs the DSI for asset discovery across system boundaries. MHHS needs the communications infrastructure that VWAN is modernising. Digital twins need the CIM-standardised network models that the LTDS programme is delivering. AI applications need consistent, high-quality data from all of these sources. If any single programme fails or is significantly delayed, the others are affected through these dependency chains.

The Clean Power 2030 target, decarbonising GB's electricity system by 2030, depends on data infrastructure that does not yet fully exist. It requires knowing where flexibility assets are (FMAR), settling them accurately (MHHS), managing consent for data sharing (CCS), enabling consumer participation through Smart Data, and coordinating across a complex multi-actor system (DSI). The data infrastructure is not a nice-to-have for decarbonisation. It is a prerequisite.

The risk is not that any individual programme fails outright. The risk is that programmes deliver on slightly different timescales, with slightly different interface specifications, and that the integration gaps between them persist longer than planned. In the meantime, the market participants who need to use these systems will continue to build workarounds, creating technical debt that becomes progressively harder to unwind. Programme coordination is where the real risk lies: DSI interfaces must match consent services, FMAR data must be usable by digital twins, and MHHS data must be discoverable under the right access controls.

Digitalisation of the energy system is not a goal in itself - it is the infrastructure that makes decarbonisation possible at the speed and scale required.

Energy Data Taskforce, A Strategy for a Modern Digitalised Energy System (2019)

This framing from the Energy Data Taskforce establishes data infrastructure as a prerequisite for clean power, not an optional enhancement. The seven transformation programmes taught here are not digitalisation for its own sake; they are operational foundations for a flexible, decarbonised electricity system.

Common misconception

AI will solve all energy system data challenges by 2030.

AI is a powerful tool but it depends entirely on the quality and availability of underlying data. A machine learning model trained on inconsistent, fragmented, or incomplete data will produce inconsistent, fragmented, or incomplete predictions. The seven transformation programmes are necessary precisely because they create the data foundations that AI requires. Without MHHS, CIM standardisation, and the DSI, AI applications will remain limited to the domains where clean data already exists.

Check your understanding

Approximately how many meter readings per year will MHHS generate when fully operational?

Core distinctions

  • Seven transformation programmes are running in parallel: DIP (live market messaging), MHHS (half-hourly settlement), CCS (consumer consent), DSI (cross-sector discovery and trust), FMAR (flexibility market arrangements), Smart Data (customer-authorised data sharing), and Clean Power 2030 (the policy destination).
  • MHHS will generate approximately 500 billion meter readings per year, requiring fundamentally different data infrastructure from current batch processing.
  • AI applications including demand forecasting, carbon intensity prediction (14 regions, 96-hour forecast), and predictive maintenance are already operational and growing.
  • Digital twins of the GB electricity network are emerging incrementally, built on CIM-standardised network models with live operational data overlays.
  • The Clean Power 2030 target depends on data infrastructure: the energy transition is as much a data challenge as an engineering one. Programme interdependency is the primary risk.

Standards and sources cited in this module

  1. NESO. Data Sharing Infrastructure

    Programme scope and journey so far

    Primary source for the DSI programme description, socio-technical architecture, and staged development status.

  2. Elexon. Market-wide Half Hourly Settlement and the BSC

    Data volume and settlement impact

    Source for the 500 billion readings per year estimate and the transition from profiled to actual half-hourly settlement.

  3. National Grid ESO. Carbon Intensity API Documentation

    API specification, regional granularity, and forecast methodology

    Source for the 14-region, 30-minute granularity carbon intensity data and 96-hour forecast capability that demonstrates operational AI in GB energy.

Module 2 of 5 in Energy System Data