The big picture: why energy data matters

Every time you switch on a light, boil a kettle, or charge your phone, a chain of events happens that you never see. Electricity is generated at a power station or wind farm, travels through high-voltage transmission lines, steps down through local distribution networks, and arrives at your home through your meter.

Alongside the electricity, there is an invisible river of data. Your smart meter records how much energy you use every 30 minutes. That data travels to the Data Communications Company (DCC), then onwards to your energy supplier so they can bill you, and to Elexon so the national electricity market can be settled correctly. A second river runs in parallel from substation SCADA telemetry to the network operator's planning systems and on into grid investment decisions. Both rivers depend on the same regulator and the same industry codes.

In everyday language, energy system data is any information created because energy is being generated, transported, or consumed. It includes everything from smart meter readings to the technical specifications of a transformer in a substation, from the price of electricity on the wholesale market to the geographic coordinates of an underground cable.

In the official regulatory world, the definition is precise. Ofgem, the energy regulator, changed it formally on 8 October 2024 so that the regulated scope is no longer limited to data needed for the operation of the system.

The definition matters because it determines which data falls under Ofgem's Data Best Practice Guidance, the rules that govern how energy data must be managed. Under the old definition, many valuable datasets were outside scope. The new definition brings them all in.

Great Britain's energy system generates an extraordinary volume of data. At the end of March 2026, 72 percent of all meters operated in smart or advanced mode, according to DESNZ Q1 2026 statistics. Five scale indicators bound the engineering challenge:

• Over 41 million smart and advanced meters were in homes and small businesses at the end of March 2026. Over 38 million were operating in smart mode or classified as advanced. Separately, Elexon says MHHS processing will handle up to 500 billion half-hourly meter readings per year from May 2027.
• Thousands of SCADA-monitored substations, each sampled every 2 to 10 seconds, produce approximately 69 million data points per hour for a large DNO. This data flows over air-gapped Operational Technology networks, physically separated from the internet.
• About 20 gas chromatographs at National Transmission System entry points continuously measure the exact calorific value of gas flowing into the national pipeline system, determining how every gas bill in GB is calculated.
• 13 categories covering 87 distinct data types, each with a defined producer, consumer, governing instrument, and sensitivity classification. This course groups the estate this way as a teaching map, not an industry standard.
• Seven major industry codes (BSC, REC, SEC, Grid Code, DCUSA, CUSC, UNC), each governing specific aspects of data creation, flow, and use.

Managing this volume reliably, securely, and accurately is one of the greatest data engineering challenges in UK infrastructure.

Scale on its own creates no obligations. What turns Ofgem's Data Best Practice Guidance from advice into a duty is the licence. Network licences carry a standard special condition on data and digitalisation which requires the licensee to collect, manage and share data in line with the guidance and to publish a Digitalisation Strategy and Action Plan. Data governance is therefore a regulated obligation for network operators rather than a voluntary commitment, and non-compliance can trigger formal enforcement action. That is also why the October 2024 change to the definition mattered so much: widening what counts as energy system data widens what the licence condition reaches. The rulebook module in the applied stage works through the hierarchy in full, and the Ofgem guidance page listed at the end of this module is the primary text.

The GB energy data system is defined by genuine, unresolved tensions. Understanding these helps a learner read every other module in this course in context. These are not academic debates. They affect how data flows, who can access it, and what happens to personal privacy.

Tension 1: Open data vs security risk

Ofgem's Data Best Practice Guidance presumes data should be open by default. SCADA telemetry from substations, however, is a national security concern. The 2015 Ukrainian power grid cyber attack showed what happens when operational data reaches the wrong hands. The Data Triage Playbook tries to square this circle, but the line between "open for innovation" and "restricted for security" remains contested.

Tension 2: Centralisation vs fragmentation

Nordic countries use a single data hub. Norway's Elhub handles all meter data in one hop. GB uses several separate platforms, including DCC, Elexon DIP, ElectraLink DTS, REC services, and NESO data services. NESO is the interim coordinator for the Data Sharing Infrastructure, which is being developed as common governance, processes and technology for secure data and model sharing. A single hub is simpler, but creates a single point of failure and raises governance questions about who controls it.

Tension 3: Consumer consent vs system need

Half-hourly smart meter data is personal data when it can be linked to an identifiable household because it can reveal routines and occupancy. Settlement, billing, network planning and customer-authorised services may each rely on different lawful bases, safeguards and access routes. The Consumer Consent Service under development by RECCo is aimed at standardising consent for consumer-directed sharing with trusted third parties.

Tension 4: Innovation speed vs regulatory pace

AI and digital twins can transform grid planning today. RIIO price controls operate on 5-year cycles, and code modification takes 12 to 18 months. The system data infrastructure risks being permanently one regulatory cycle behind the technology curve. RIIO-3 creates stronger standards for data and digitalisation, but programme coordination still has to keep pace with Clean Power 2030 delivery.

Tension 5: National standards vs local variation

CIM provides an international data model (IEC 61970). Each of the 14 DNOs implements it differently, with varying maturity levels. The LTDSprogramme is closing the gap. Stage 1.3 completed November 2025, Stage 2 targets May 2026, Stage 3 targets November 2026. Full interoperabilityremains years away.

This course has 15 modules across three stages:

1. Foundations (Modules 1 to 5): what energy data is, who produces it, how the physical network generates it, the complete 87-type taxonomy, and smart meters.
2. Applied (Modules 6 to 10): the data lifecycle from meter to market, the regulatory hierarchy, data governance, privacy rights, and settlement mechanics.
3. Practice and Strategy (Modules 11 to 15): international comparison, transformation programmes, digitalisation governance, CIM interoperability, and strategic planning.

Each module starts with a learning contract (the three outcomes you will achieve), includes inline knowledge checks, and ends with key takeaways. Each stage has a practice assessment you can use before attempting the timed stage test.