Data quality and meaning
By the end of this module you will be able to:
- Define and distinguish the six core GOV.UK and DAMA UK data quality dimensions
- Apply a data quality dimension to assess a described dataset scenario
- Explain the difference between data quality and data fitness for purpose
- Describe a master data management approach for a given quality problem
Six quality dimensions and the rule, gate, repair, owner for each
Each of the six dimensions carries its own rule, gate, repair and named owner, so a failure has somewhere to go: validity is tested against the schema, rejected at the producer, and never reaches the consumer who would have used it.
Quality is a matrix of six dimensions, each with a measurable rule, a gate, a repair, and a named owner. The UK Government Data Quality Framework Annex A supplies the six operational dimensions; ISO 8000-8:2015 supplies measurement concepts rather than this six-item list.
Anatomy of a useful data quality rule
Field, expectation, threshold, action and evidence each do a different job, so dropping one does not weaken the rule evenly: without a threshold it cannot decide, without an action it fails silently, and without evidence it cannot be audited.
A quality rule has five parts: field, expectation, threshold, action, and evidence. Drop any one and the rule cannot run or cannot be audited. DAMA-DMBOK 2 and UK GDQF Annex A both spec this anatomy.
Quality remediation as a five-step learning loop
Detect, triage, repair at source and verify by re-running the rule all fix one instance; only the learn step changes the next cycle, so a team that stops at verify meets the same defect again every reporting period.
Quality remediation is a five-step loop: detect, triage, repair, verify, learn. The learn step feeds the next detection rule. Dropping it turns quality work into a treadmill of repeated defects. DAMA-DMBOK 2 names the same shape.
Duplicate records inflated hospital death counts by 552.
In 2019, NHS England published hospital mortality statistics that incorrectly attributed 552 additional deaths to a particular hospital trust. The cause was duplicate patient records in the underlying data pipeline: patients treated at multiple hospitals had separate records that were not de-duplicated before analysis.
The trust faced reputational damage before the error was identified and corrected. Data quality is not an abstract concept, and the costs of poor quality extend well beyond spreadsheet errors.
The previous module covered how to present data clearly. This module covers how to ensure the data is worth presenting in the first place.
No one deliberately falsified data. Yet a hospital trust faced reputational damage because of duplicate records. Is bad data quality always someone's fault?
The NHS case illustrates a principle that runs through every data role: the quality of decisions cannot exceed the quality of the data they rely on. The Government Data Quality Framework is blunt about the consequences: poor-quality data leads to service failure, poor decisions, and workarounds that hide the root cause. The actual costs include wasted effort, incorrect decisions, compliance failures, and reputational damage.
The opening incident is a uniqueness failure with real operational consequences. The first task is to define quality in a way that links a dataset to the decision it is meant to support.
8.1 What data quality means
Data quality is the degree to which data is fit for the intended use of its consumers. The critical phrase is "fit for purpose." A dataset with 5% address errors may be acceptable for postal marketing but unacceptable for regulatory compliance reporting.
ISO 8000-8:2015 explains the concepts and measurement prerequisites for information and data quality, including syntactic, semantic, and pragmatic levels. GOV.UK's Government Data Quality Framework, drawing on DAMA UK, uses six core dimensions for practical assessment: completeness, uniqueness, consistency, timeliness, validity, and accuracy.
“Data quality: the degree to which a set of inherent characteristics of data fulfils requirements.”
ISO 8000-8:2015, Data quality - Clause 3.1, definition of data quality
The ISO definition deliberately ties quality to requirements. A dataset is not 'high quality' or 'low quality' in the abstract. Quality is always measured against a stated purpose. This is why the same dataset can be adequate for one team and inadequate for another.
Once quality is tied to requirements, it needs measurable checks. The six GOV.UK and DAMA UK dimensions give a practical vocabulary for finding different kinds of failure.
8.2 The six dimensions of data quality
- Accuracy: the degree to which data correctly reflects the real-world entity or event it represents. A customer record with a misspelled surname is inaccurate. An inventory record showing 50 units when 47 are physically present is inaccurate. Accuracy can only be assessed by reference to a ground truth.
- Completeness: the degree to which required data values are present. A patient record with no date of birth is incomplete. Measured as the proportion of non-null required fields. 100% completeness is not always achievable: some data is genuinely unknown at collection time.
- Consistency: the degree to which data values agree across related records or systems. A customer marked "Active" in the CRM but "Closed" in the billing system is an inconsistency. These failures commonly arise from multiple systems holding overlapping data without synchronisation.
- Timeliness: the degree to which data is current and available when needed. A postcode file updated annually may be timely for annual reporting but stale for a logistics system routing same-day deliveries.
- Validity: the degree to which data conforms to defined rules, formats, or ranges. A date of birth of "31/02/2001" is invalid (February has no 31st day). Validity is testable against schema constraints.
- Uniqueness: the degree to which records are not duplicated. The NHS mortality incident was a uniqueness failure: the same patients appeared multiple times, inflating counts.
Common misconception
“If the data is complete, it must be high quality.”
Completeness is just one quality dimension. A dataset can be 100% complete (no empty fields) while containing inaccurate values, invalid formats, and duplicate records. A CRM with every field filled but 23% duplicate customers has a completeness score of 100% and a severe uniqueness problem. All quality dimensions must be assessed independently.
A dimension is only useful when it becomes a rule, a target, and a threshold. Measurement turns quality language into evidence that can be monitored and challenged.
8.3 Measuring data quality in practice
Data quality measurement begins with profiling: automated analysis that characterises a dataset's contents. A profiling exercise typically measures completeness (null counts per field), validity (format and constraint failures), uniqueness (duplicate record counts), and pattern conformance (values matching expected patterns such as postcodes or phone numbers).
GOV.UK's 2026 Data Quality Issues Framework makes this operational: each critical data asset needs rules that define what good data looks like, targets for those rules, performance bands for below-target data, and a priority based on the purpose and impact of failure. That prevents a common mistake where every issue is labelled high priority and nothing is fixed first.
Tools such as Great Expectations (open source), Soda Core, and dbt tests automate profiling as part of data pipeline runs. The output is a data quality report that flags violations against defined thresholds.
Measurement exposes defects, but it does not decide whether a dataset is usable. Fitness for purpose asks whether the measured quality is sufficient for a specific decision.
8.4 Data quality versus fitness for purpose
A dataset can score well on all quality dimensions and still be unfit for a specific purpose. A marketing database with complete, accurate, consistent, timely, valid, and unique records may be entirely unfit for clinical research because it lacks medical context, provenance documentation, and consent records.
Quality assessments should always begin with a use case definition: what decisions will this data support, and what threshold does each dimension need to meet for those decisions to be reliable?
Common misconception
“Data quality is IT's problem to fix.”
Data quality failures originate in business processes: manual data entry without validation, systems that allow free-text where a lookup table should be used, and workflows that create duplicate records when updates should be applied. IT can build validation rules and profiling tools, but the processes that generate poor data belong to business teams. Fixing quality requires business process change, not just technical remediation.
Many quality failures come from shared entities being recreated in different systems. Master data management is the governance pattern for controlling those entities at source.
8.5 Master data management
Master data management (MDM) is the set of processes, governance, policies, and tools used to maintain a single, authoritative, and consistent version of key shared data entities: customers, products, employees, and locations.
In practice, MDM involves four steps:
- Identifying the master data domains (customer, product, supplier, location)
- Designating a system of record for each domain: the authoritative source
- Implementing matching and merging rules to de-duplicate records across systems
- Establishing governance to control who can create, update, or retire master records
A UK financial services firm audited its CRM in 2021 and found that 23% of customer records were duplicates, created when customers changed contact details and were re-registered rather than updated. The downstream effects included wasted marketing spend, incorrect churn metrics, and an inflated customer base that affected investment decisions. The MDM programme to resolve this took eight months.
A retailer's database shows a product weight of 2.5kg. The physical product weighs 1.8kg. The record also has a null value in the 'supplier_code' field, which is marked as required. Which quality dimensions are affected?
A local authority uses address data from its housing register, last updated 18 months ago, to send council tax reminder letters. A high proportion of letters are returned as undeliverable. Which quality dimension is the primary issue?
Your organisation runs a one-time data cleansing project that corrects 100,000 records. Six months later, quality metrics show degradation returning to pre-cleansing levels. What went wrong?
Core distinctions
- ISO 8000-8 defines data quality by reference to requirements and provides concepts for syntactic, semantic, and pragmatic measurement. GOV.UK and DAMA UK provide the six core dimensions used in this module.
- Quality is always relative to intended use. A dataset can score well on every dimension and still be unfit for a specific purpose if it lacks the context, provenance, or consent records that purpose requires.
- A data quality action plan should identify critical data, set quality rules, assess current performance, prioritise improvements, fix root causes, report results, and repeat measurement over time.
- Master data management (MDM) prevents conflicting versions of key entities. It requires governance and ongoing process change, not a one-time cleansing exercise.
- DAMA-DMBOK frames data quality as a programme, not a project. Fixing processes that create poor data is more important than cleansing the data they have already produced.
Standards and sources cited in this module
Concepts and measuring
Defines core data quality concepts and measurement prerequisites, including syntactic, semantic, and pragmatic quality levels.
ISO/IEC 25012:2008, Data quality model
Full standard
Broader quality framework distinguishing inherent quality (accuracy, completeness) from system-dependent quality (availability, recoverability).
GOV.UK Government Data Quality Framework
Data quality dimensions and lifecycle guidance
UK public sector guidance that names the six core dimensions and frames quality as fit for purpose.
GOV.UK Data Quality Action Plan implementation guide
Seven-step DQAP process
Current guidance for turning quality dimensions into rules, targets, root-cause action, reporting, and repeat monitoring.
GOV.UK Data Quality Issues Framework
Rules, targets, performance bands, and prioritisation
Current framework for identifying, prioritising, and acting on data quality issues for critical data assets.
Module 7 of 52 · Foundations