Data security essentials

Three properties describe what security is trying to preserve. Confidentiality means that only people and systems with a reason to read the data can read it. Integrity means the values are what they were meant to be, that unauthorised change is prevented or detected, and that the record of change is trustworthy. Availability means the data is there when a legitimate process asks for it, within the time that process can tolerate.

The useful move is not to recite the three but to weight them for a specific dataset, because the weighting drives the spend. A payroll table is dominated by confidentiality: a leak is career-damaging for every person in it, while an hour of downtime is an inconvenience. A billing meter archive is dominated by integrity: a silently corrupted reading produces an invoice that is legally wrong and that somebody will eventually have to unwind across thousands of accounts. A public timetable feed is dominated by availability and integrity together, and has almost no confidentiality requirement at all, because the whole point is that anybody can read it.

Getting this wrong in either direction is expensive. Over-weighting confidentiality on open reference data produces an approval process that stops the data being used for the purpose it was published for. Under-weighting integrity on a financial feed produces a system where nobody can say whether last quarter's numbers were right, and where the only honest answer to an auditor is that the pipeline has no checks that would have caught the change.

UK data protection law folds all three into a single principle. The principle is named integrity and confidentiality, but its text covers accidental loss and destruction, which is availability by another name.

Encryption at rest protects data while it is sitting on disk, in object storage or in a backup. The threat it answers is physical or accidental exposure of the stored bytes: a laptop left on a train, a decommissioned drive that was never wiped, a storage bucket left open to the internet, a backup tape in a courier van. In each of those cases the attacker gets the file and not the key, so the file is noise.

Encryption in transit protects data while it moves, normally with Transport Layer Security. The threat it answers is interception and tampering on the network path. This matters more on a data platform than people expect, because a pipeline is a long chain of hops between services, and each unprotected hop is a place where traffic can be read or altered without either end noticing. Internal traffic counts. The idea that the private network is safe stopped being defensible when the private network became a virtual network shared with everything else the organisation runs.

Now the honest part. Neither control stops the attack in the figure above. A stolen analyst session arrives at the platform holding valid credentials. The storage layer decrypts for that session exactly as it decrypts for a legitimate one, because from the storage layer's point of view there is no difference. Encryption is not a judgement about who is asking. It is a property of the stored and transmitted form. A credential attack walks straight past it, and the control that reduces the damage is the one further up the stack that limits what a permitted reader can see.

Encryption fails in practice through key management rather than through broken ciphers. The recurring failure modes are worth naming individually. Keys stored beside the data they protect, so that whoever takes the data takes the key. Keys held entirely by the service provider, so that a compulsory access request served on the provider reaches plaintext without the customer ever being told. No rotation, so a key exposed years ago still opens everything written since. No revocation path, so the response to a suspected compromise is a project rather than an operation. And the opposite failure, which is destruction: a key deleted without a tested recovery path takes the data with it, permanently, and no amount of backup of the ciphertext helps.

The practical question to ask a platform team is not whether encryption is enabled. It is who can compel decryption, how quickly a key can be rotated, and what has ever been restored from an encrypted backup in a drill. Those three answers tell you whether the encryption is a control or a diagram.

Role-based access control attaches permissions to roles and grants roles to people. What someone may do follows from the job they hold, not from a request they once made. Its strength is administrability: there is a finite set of roles, a review can ask what each role can now reach, and a leaver loses everything at once when the role is removed. Its weakness is drift. Roles accumulate permissions because adding one to an existing role is always easier than arguing for a new role, and after three years the analyst role can read things nobody would grant an analyst today.

Attribute-based access control decides each request by evaluating attributes of the requester, the data, the action and the context against a policy. Clearance level, data classification, purpose of processing, project membership, country of access and time of day can all combine without a role being defined for every permutation. Its strength is expressiveness: rules that would need hundreds of roles collapse into a handful of policies. Its weakness is that a policy set is harder to reason about than a role list, and a question as simple as which people can currently read the payments table may only be answerable by evaluating the policy against every possible requester.

The choice is not ideological. Use roles when the meaningful distinctions line up with jobs and the population of jobs is small and stable. Move to attributes when the decision genuinely depends on something about the data or the context that a job title does not capture, which in a data platform is usually classification, purpose, or which project a request belongs to. Most mature estates run both: roles carry the coarse grant, attributes carry the conditions on top of it.

Placement matters as much as model. Row level security attaches policies to the table itself, so the database applies the restriction to ordinary queries whichever tool issued them. The PostgreSQL implementation makes the failure mode clear: tables have no policies by default and are equally readable by anyone with table privileges, but once row security is enabled with no policy present, the default is deny. It is equally clear about who sits outside the mechanism, and that is the part a reviewer has to check. Superusers and roles carrying the BYPASSRLS attribute always bypass row security, and the table owner normally bypasses it too unless the owner opts in with ALTER TABLE ... FORCE ROW LEVEL SECURITY. A policy therefore binds the roles that analysts and services actually connect as, which is an argument for those roles owning nothing and holding no bypass attribute. Column level controls do the equivalent job sideways, refusing or masking a field rather than a record.

The alternative, filtering in the reporting layer, is a convention rather than a control. It holds until somebody connects a notebook, a scheduled extract or a new dashboard directly to the warehouse, at which point the filter that lived in the report is simply absent. A policy held in the engine is the thing that actually binds, and it is also the thing you can show an auditor without a demonstration.

The access matrix figure above is a discipline rather than a diagram. Roles across the top, assets down the side, and every cell filled with none, masked or full plus the reason for the answer. Filling it exposes two things a policy document hides. First, that no row and no column is uniform: the support agent has full access to the customer table one record at a time and no access at all to model features, while the analyst has the opposite shape. Second, that an unfilled cell is not a neutral state. In a real system an undecided pairing defaults to whatever the underlying grant allows, which is almost always more than anyone intended.

Where research or third-party access is involved, the Five Safes framework gives a broader frame than access control alone. As the UK Data Service sets it out for its own secure lab, it spreads control across safe projects, safe people, safe data, safe settings and safe outputs, which explains why a secure environment can work with data that would be far too disclosive to hand over, and why relaxing any one of the five has to be paid for in the others.

None of this is inspectable after the fact without audit logging. A chronological record of sign-ins, reads, changes, exports and administrative actions is what turns access control from something that exists into something that can be shown to have worked. Without it an incident becomes an argument about what probably happened.

Data masking removes a field or replaces it with a stand-in value, so the original cannot be read. A phone number becomes a run of asterisks, a surname becomes a generated pseudonym, a date of birth becomes a year. The NIST glossary fixes the defining property in the definition itself: the removal or replacement is done in a way that leaves no analytic utility in the value, and the examples it gives are asterisks over a phone number and a randomly generated pseudonym. That is the trade: the protection and the loss are the same act. Masking is most often the only thing standing between a test environment and real customer records, which is precisely the setting where nobody reviews it.

Tokenisationreplaces a sensitive value with a surrogate and keeps the mapping back to the original in a separate protected system. The token travels through applications that never need the real value, and reversing it is a controlled operation performed by the token service. The PCI Security Standards Council's guidance sets the security bar plainly: recovering the original value must not be computationally feasible from a token, or from any number of tokens.

The distinction that matters is reversibility, and what follows from it. Masking destroys the value at the point of display or copy, so a masked field cannot be turned back and cannot support the analysis the original supported. Tokenisation preserves the value somewhere and preserves referential behaviour: the same input yields the same token, so joins and counts still work across systems that hold only tokens. What tokenisation does is relocate the secret. The sensitive value now lives in one hardened vault instead of forty applications, which shrinks the audit surface of the estate rather than reducing the risk in any single place. If the vault falls, everything falls.

The non-production case is where the choice bites hardest. A test environment needs data that behaves like production: the same distributions, the same edge cases, the same referential structure across tables. Masking gives you that structure while destroying the values, which is usually the right answer, and the failure mode is partial application, where nine columns are masked and the tenth was added last sprint by someone who did not know the masking rules existed. Tokenisation gives you structure and reversibility, which is more than a test environment should have, so the token service is normally scoped so that non-production systems can hold tokens and cannot de-tokenise. Deciding this per environment, and enforcing it in the provisioning process rather than in a wiki page, is the difference between a control and an intention.

Neither technique is anonymisation. A tokenised customer identifier still singles out an individual and is therefore still personal data, because the organisation holds the means to reverse it. A masked dataset may or may not be personal data depending on what is left and what else can be joined to it. That assessment belongs with anonymisation and the motivated intruder test, not with a product datasheet.

The UK security duty for personal data sits in two places. Article 5(1)(f) states the principle, and Article 32 turns it into an obligation on both the controller and the processor to implement measures appropriate to the risk. Appropriate is the statutory word, and the article says what it is judged against: the state of the art, the cost of implementation, and the nature, scope, context and purposes of the processing.

Around that legal core sit the frameworks a data team is measured against in practice. NIST's Cybersecurity Framework 2.0 is the common language for organising and reporting security work, and its 2024 revision added a governance function, which pulled risk ownership and policy out of the technical functions and gave them their own home. ISO/IEC 27001 supplies the certifiable management system, with information deletion sitting among its Annex A controls. ISO/IEC 27701 carries the same evidence discipline into privacy, and its status changed with the 2025 revision published in October of that year: the 2019 edition was written as an extension to ISO/IEC 27001 and 27002, whereas the 2025 edition is a standalone privacy information management system standard, so an organisation can be certified against it without holding an information security management system first. None of these frameworks tells you which controls to buy. They tell you how to decide, record the decision, and show that the decision was reviewed.

The horizon item is cryptographic. Much of the public key cryptography in use today rests on mathematical problems that a sufficiently capable quantum computer could solve. Post-quantum cryptography replaces those algorithms with ones built to resist that attack. NIST released the first standards in August 2024: FIPS 203 for key encapsulation, and FIPS 204 and FIPS 205 for digital signatures.

The reason this belongs in a data course rather than only in a cryptography one is the shape of the risk. Encrypted traffic captured today can be stored until a machine exists that can open it, so the exposure is set by how long the data must stay confidential rather than by when the machine arrives. For a session cookie that is irrelevant. For genetic data, adoption records, national security material or thirty-year commercial contracts it is not. The practical first step is not a migration but an inventory: which datasets have a confidentiality requirement that outlives the current algorithms, and where does the cryptography protecting them actually live?