Standards and interoperability
Why data standards exist, who creates them, and which standards govern the most common data exchange scenarios from dates and geography to healthcare records.
By the end of this module you will be able to:
- Distinguish de jure from de facto data standards with examples
- Apply ISO 8601 date formatting rules to avoid common ambiguity errors
- Identify the correct standard for at least three common data exchange scenarios
- Explain why the NHS Test and Trace file-size incident occurred and how a controlled exchange standard would have reduced the risk
What real interoperability needs, layer by layer
Each layer assumes the one below it is already settled, so two systems that agree on syntax and structure have only agreed how to parse each other, and without the semantics layer they still disagree about what a customer is.
Two systems agreeing on JSON do not interoperate; they parse the same bytes. Real interoperability needs syntax, structure, identifiers, semantics, and governance, in that order. CSVW and JSON Schema describe payload structure; DCAT describes catalogue metadata for discovery.
Why interoperability needs syntactic and semantic layers
Bytes that parse, a schema that is understood and identifiers that resolve are all still syntax, and only the top layer agrees what a word means, so two systems reading identical JSON can still disagree about who counts as a customer.
Interoperability has two layers, not one. Syntactic interoperability means the receiver can parse the bytes; semantic interoperability means the receiver and sender mean the same thing by customer. ISO/IEC 11179-1:2023 §3 separates the two; W3C DWBP §10 mandates a shared vocabulary for semantic alignment.
Five binding clauses every API contract should carry
Endpoint, schema, error model, rate limits and deprecation policy are the five clauses one API contract binds. The error model is the one most often left out, which leaves consumers retrying on everything and producing duplicate writes.
An API contract binds the provider to the consumer in five clauses: endpoint, schema, error model, rate limits, deprecation policy. OpenAPI 3.2.0 carries the first three machine-readably; ODCS v3.1.0 adds the operational ones.
How a governed crosswalk lets two systems share one subject
Neither local identifier reaches the joined view directly; both pass through the versioned crosswalk that maps them to one canonical id, so a crosswalk left unversioned lets a reused key join two different subjects unnoticed.
Two systems can hold the same customer in two different local identifiers and never join them. A governed crosswalk to a canonical identifier is the only thing that lets them meet. W3C DWBP §8.10 mandates persistent identifiers; ISO/IEC 11179-1:2023 §4.3 scopes them.
Picking the interoperability standard from the data shape
The shape of the data picks the standard, rows to CSVW, trees to JSON Schema and OpenAPI, triples to RDF and SHACL, so a graph pushed into CSV loses its relationships and a tree pushed into rows repeats its parents.
Standards selection is a three-question decision: data shape, consumer, governance. Tabular -> W3C CSVW. Nested -> JSON Schema 2020-12 + OpenAPI 3.2.0. Graph -> W3C RDF + SHACL; catalogue metadata -> DCAT-3. Picking the wrong standard locks in a wrong-shaped exchange.
NHS Test and Trace: 15,841 COVID-19 results delayed by a data load file-size failure
On 4 October 2020, Public Health England said that 15,841 COVID-19 positive test results from 25 September to 2 October were not included in reported daily case figures because of a technical issue in the data load process. The official update said some files containing positive test results exceeded the maximum file size used by the loading process.
The failure was not just a tool choice. It was a standards and control failure: a critical pipeline accepted a batch whose size, format, and error handling were not governed tightly enough. A controlled exchange standard with declared schema, capacity checks, load validation, and rejection alerts would have made the failure visible before national reporting and contact-tracing delays.
A critical reporting pipeline delayed 15,841 positive COVID-19 test results. How does a weak exchange control become a public health failure?
Why data standards exist and how they are classified
A data standard is a documented agreement specifying how data should be structured, formatted, encoded, or exchanged. Without standards, every system that exchanges data must negotiate a custom format with every other system. With standards, any two conforming systems can exchange data without prior negotiation. This is the economic argument for standardisation.
De jure standards are formally published and ratified by a recognised body: ISO (International Organisation for Standardisation), IEC (International Electrotechnical Commission), W3C (World Wide Web Consortium), or IETF (Internet Engineering Task Force). Compliance may be voluntary or mandated by regulation.
De facto standards achieve widespread adoption through market dominance or practical necessity, without formal standardisation. JSON began as a de facto standard (derived from JavaScript object syntax) before being formalised as RFC 4627 in 2006, updated as RFC 8259 in 2017. PDF was a de facto standard before ISO standardisation as ISO 32000.
Knowing which body ratified a standard says nothing about how far a given exchange actually works. Interoperability is better read as three levels, each of which can fail on its own.
Syntactic, semantic, and operational interoperability
Interoperability has levels. Syntactic interoperability means systems can parse the same format, such as valid JSON or CSV. Semantic interoperability means they attach the same meaning to the fields, codes, and units. Operational interoperability means the exchange works in a real process with versioning, validation, error handling, ownership, and agreed support. Most expensive failures happen because teams achieved syntax and mistook it for meaning or operational readiness.
The practical test is simple: can two systems exchange a value, parse it, attach the same meaning to it, and handle failure without manual rescue?
Dates and delimited files: ISO 8601 and RFC 4180
ISO 8601:2019 (dates and times) defines the internationally unambiguous date format as YYYY-MM-DD. The date "01/02/03" is ambiguous: it could mean 1 February 2003 (UK/European format), 2 January 2003 (US format), or 3 February 2001 (ISO with 2-digit year). ISO 8601 eliminates all ambiguity. Full datetime: 2024-06-14T09:30:00Z (Z indicates UTC). The date format DD/MM/YYYY is standard in the UK; MM/DD/YYYY in the US. Any data exchange that does not specify which convention is in use creates ambiguity for all dates where the day value is 12 or below.
RFC 4180 (CSV format) specifies: comma as field delimiter, CRLF as line terminator, double-quote for escaping fields containing commas or newlines, and the first line as an optional header row. Many CSV files in the wild do not conform, using semicolons (common in European locales where commas are decimal separators) or other delimiters.
ISO 8601 and RFC 4180 apply wherever data is exchanged. Geography and health care each add a domain standard on top, with its own rules about what the fields mean.
Geography and health data: GeoJSON and HL7 FHIR
RFC 7946 (GeoJSON) defines a JSON-based format for geographic features using WGS 84 coordinates. Critical: GeoJSON specifies longitude before latitude: [longitude, latitude]. Google Maps uses (lat, lng) order. Mixing these produces points plotted in the ocean or on the wrong continent. This is the most common GeoJSON mistake.
HL7 FHIR R4 (health data) defines resources (patient, observation, medication, condition, appointment) as structured JSON or XML documents with a RESTful API profile. NHS England uses FHIR in health and care API standards because it gives suppliers a common model for exchanging patient, observation, medication, appointment, and condition data without inventing a new format for each integration.
Clinical terminologies and classifications are two different things that look alike from outside, and health data uses both at once. NHS England describes SNOMED CT as a structured clinical vocabulary for capturing detailed clinical information in the patient record, ICD-10 as a classification for diagnostic information about diseases, injuries and cause of death, and dm plus d as a dictionary of codes representing the medicines and devices in use across the NHS. A terminology is granular and written for the clinician recording care. A classification groups records so they can be counted, which is what national statistics need. Neither is a worse version of the other, so a proposal to standardise on one of them asks half the estate to stop doing its job. The workable answer is an interoperability layer that maps between them using published crosswalks and carries the result over FHIR, keeping each source record in the coding system it was written in.
“To promote the interoperability among datasets it is important to adopt data vocabularies and standards.”
W3C Data on the Web Best Practices - Section 4, Context
Common misconception
“Using standard date formats is a minor formatting preference, not a correctness requirement.”
Date format ambiguity is a data correctness issue, not a style preference. '01/02/03' means three different dates depending on whether you are in the UK, the US, or using ISO short form. Any date where the day value is 12 or below is ambiguous in non-ISO formats. In a dataset combining records from UK and US teams, ambiguous dates will be silently mis-parsed. The systemic fix is ISO 8601 (YYYY-MM-DD) in all data files and APIs, not case-by-case manual checking. Requiring ISO 8601 at source prevents ambiguity; accepting non-standard formats creates it.
A government agency receives data from 12 organisations. Organisation A submits dates as '14/06/2024', Organisation B as '06/14/2024', and Organisation C as '2024-06-14'. The agency must merge all three files. Which approach best resolves the ambiguity?
A developer creates a GeoJSON file marking the location of 50 air quality monitoring stations in London. When plotted on a map, all stations appear in the Indian Ocean near the equator. What is the most likely cause?
Core distinctions
- Data standards exist to enable interoperability: any two conforming systems can exchange data without custom integration. The economic case is compelling at scale.
- De jure standards are formally ratified by ISO, IETF, W3C, and similar bodies. De facto standards emerge from widespread adoption and may be later formalised (JSON: de facto 2001, ratified as RFC 8259 in 2017).
- ISO 8601 (YYYY-MM-DD), RFC 4180 (CSV), RFC 7946 (GeoJSON), and HL7 FHIR R4 (healthcare) are common standards for dates, tabular files, geography, and health data exchange.
- GeoJSON specifies (longitude, latitude) order, the opposite of many map application conventions. This single ambiguity causes the most common GeoJSON error.
- The NHS Test and Trace reporting failure shows why critical pipelines need declared formats, size limits, validation, and load alerts. A standard is not only a file format; it is also an operational contract.
Standards solve the format problem. The next module asks a different question: once data exists in a standard format, who can use it and under what conditions? Open data, the FAIR principles, and licensing frameworks determine whether data is legally, technically, and practically reusable.
Standards and sources cited in this module
ISO 8601:2019: Date and time format
Authoritative international standard for date and time representation: YYYY-MM-DD and full datetime formats.
RFC 4180 (IETF, 2005): Common Format for CSV Files
CSV format specification: delimiter, quoting, line terminator, and header row conventions.
RFC 7946 (IETF, 2016): GeoJSON Format
Section 3.1: Position. GeoJSON coordinate order specification (longitude, latitude).
Fast Healthcare Interoperability Resources: patient, observation, and medication resource definitions.
Public Health England: delayed reporting of COVID-19 cases (October 2020)
Official statement confirming 15,841 delayed positive cases and the maximum file-size failure in the data load process.