Privacy and privacy-enhancing technologies

Security protects systems and data from unauthorized access; privacy concerns whether data about people is collected, used and shared appropriately in the first place, even by authorized parties. A perfectly secure database can still violate privacy if it holds data that should never have been collected or uses it for purposes people did not expect. This lesson covers the principles behind privacy law, the reasons anonymization is harder than it looks, and the privacy-enhancing technologies (PETs) that let organizations learn from data while protecting individuals: generalization and k-anonymity, differential privacy, pseudonymization and data minimization, with a look at encryption-based techniques.

CMPUT-291 introduced privacy law as it affects database design, and BUS-210 covered privacy law from a business-law perspective. Here the emphasis is on the technical mechanisms and their limits.

Principles of privacy

Modern privacy law in Canada, the United States and Europe draws on the fair information principles first set out in the 1980 OECD guidelines: accountability, identifying purposes, consent, limiting collection, limiting use, disclosure and retention, accuracy, safeguards, openness, individual access and the right to challenge compliance. Privacy by design, a framework developed by Ann Cavoukian when she was Ontario's Information and Privacy Commissioner, asks engineers to build these principles into systems from the start: privacy as the default setting, embedded in design rather than added later, and protecting data across its whole life cycle.

In the United States there is no comprehensive federal privacy law. Instead, sector laws cover health information (HIPAA), education records (FERPA), children's data online (COPPA), financial data (the Gramm-Leach-Bliley Act) and credit reports (FCRA), and the FTC polices unfair or deceptive practices. About twenty states have comprehensive consumer privacy laws, led by California's CCPA as amended by the CPRA, giving rights to know, delete, correct and opt out of sale or sharing, so check which apply to your users.

Why anonymization is hard

Removing names and ID numbers does not make data anonymous. Combinations of ordinary attributes, called quasi-identifiers, can single people out. In a well-known 2000 study, Latanya Sweeney estimated that 87 percent of the US population could be uniquely identified by five-digit ZIP code, birth date and sex alone. Released datasets have repeatedly been re-identified by linking them with other public information: a state hospital discharge dataset matched against voter rolls, search logs whose queries revealed who made them, and a movie-rating dataset matched against public reviews. The lesson for engineers is that whether data is identifiable depends on what other data exists, which the publisher cannot fully know.

Laws reflect this. Canadian privacy commissioners and courts treat information as personal if there is a serious possibility that an individual could be identified, alone or combined with other available information, and US health law offers two routes to de-identification under HIPAA: removing a specified list of identifiers, or obtaining an expert determination that the risk is very small.

Generalization and k-anonymity

One classic response is k-anonymity, introduced by Sweeney and Pierangela Samarati: a release is k-anonymous if every combination of quasi-identifier values is shared by at least \( k \) records, so each person hides in a group of at least \( k \). It is achieved by generalizing values (birth year to decade, postal code to its first characters) and suppressing outliers. The example measures \( k \) for a toy release, then generalizes it.

from collections import Counter
# A toy release of clinic visits: quasi-identifiers are postal prefix, birth year and sex
rows = [("T6G", 1998, "F", "asthma"), ("T6G", 1998, "F", "flu"), ("T6G", 1997, "F", "flu"),
        ("T5K", 1985, "M", "diabetes"), ("T5K", 1985, "M", "flu"), ("T5K", 1986, "M", "asthma"),
        ("T6E", 2001, "F", "migraine"), ("T6E", 2001, "M", "flu")]
def k_of(table, key):
    groups = Counter(key(r) for r in table)
    return min(groups.values()), groups
k, groups = k_of(rows, lambda r: (r[0], r[1], r[2]))
print("raw quasi-identifiers: k =", k, "(some people are unique)")
# Generalize: birth year -> decade, postal prefix -> first two characters, drop sex
gen = [(r[0][:2], r[1] // 10 * 10, r[3]) for r in rows]
k, groups = k_of(gen, lambda r: (r[0], r[1]))
print("generalized: k =", k)
for g, n in sorted(groups.items()):
    diseases = sorted({r[2] for r in gen if (r[0], r[1]) == g})
    print(f"  group {g}: {n} people, diagnoses {diseases}")

raw quasi-identifiers: k = 1 (some people are unique)
generalized: k = 2
  group ('T5', 1980): 3 people, diagnoses ['asthma', 'diabetes', 'flu']
  group ('T6', 1990): 3 people, diagnoses ['asthma', 'flu']
  group ('T6', 2000): 2 people, diagnoses ['flu', 'migraine']

The output shows k-anonymity's weaknesses as well as its strength. Generalization raised \( k \) from 1 to 2, but look at the diagnoses: if every member of a group had the same diagnosis, knowing someone is in the group would reveal it anyway, a homogeneity problem that refinements called l-diversity and t-closeness try to address. Worse, k-anonymity says nothing about what happens when the same data is released twice with different generalizations, or combined with other sources. These limits motivated a stronger, mathematical definition.

Differential privacy

Differential privacy, introduced by Cynthia Dwork and colleagues in 2006, is a property of an algorithm rather than of a dataset. An algorithm is \( \varepsilon \)-differentially private if adding or removing any one person's data changes the probability of any output by at most a factor of \( e^{\varepsilon} \). Whatever an attacker already knows, the output reveals almost nothing more about any individual than if their record had never been included. The simplest way to achieve it for a numeric query is to add random noise drawn from a Laplace distribution whose scale is the query's sensitivity divided by \( \varepsilon \):

import numpy as np
rng = np.random.default_rng(331)
# Differential privacy: answer a counting query with Laplace noise of scale sensitivity / epsilon
true_count = 42                      # e.g. students in a course who used the student advising service
sensitivity = 1                      # one person changes a count by at most 1
for eps in (0.1, 1.0, 5.0):
    answers = true_count + rng.laplace(0, sensitivity / eps, size=10_000)
    one = true_count + rng.laplace(0, sensitivity / eps)
    err = np.mean(np.abs(answers - true_count))
    print(f"epsilon {eps:<4}: one noisy answer = {one:6.1f}; average error over 10,000 runs = {err:5.2f}")
# Repeated queries add up: k answers at epsilon each cost k * epsilon in total
print("total privacy cost of 20 queries at epsilon 0.5 =", 20 * 0.5)

epsilon 0.1 : one noisy answer =   63.9; average error over 10,000 runs =  9.93
epsilon 1.0 : one noisy answer =   39.3; average error over 10,000 runs =  1.01
epsilon 5.0 : one noisy answer =   42.1; average error over 10,000 runs =  0.19
total privacy cost of 20 queries at epsilon 0.5 = 10.0

The parameter \( \varepsilon \) is the privacy budget: small values such as 0.1 give strong protection and noisy answers, while larger values give accurate answers and weaker protection. Privacy loss adds up across queries, so a system must track the total budget and stop answering when it is spent. Differential privacy is used in practice: the US Census Bureau applied it to the 2020 census results, and Apple, Google and Microsoft use local variants to collect usage statistics from devices without learning any individual's data. Its costs are reduced accuracy, particularly for small groups, and the difficulty of choosing \( \varepsilon \).

Pseudonymization and minimization

Pseudonymization replaces direct identifiers with consistent substitutes, so records about the same person can still be linked for analysis without revealing who they are. A keyed function such as HMAC with a key held separately from the data is the right tool. An unkeyed hash is not: when the space of possible identifiers is small, such as seven-digit student numbers, anyone could compute the hash of every possible value and reverse the mapping. Pseudonymized data is still personal information under Canadian and European law, because whoever holds the key can re-identify people, but it reduces risk considerably.

import hmac, hashlib, secrets
# Pseudonymizing student IDs for a research extract
KEY = b"research-project-key; stored apart from the data, destroyed at project end"
def pseudonym(student_id: str) -> str:
    return hmac.new(KEY, student_id.encode(), hashlib.sha256).hexdigest()[:16]
ids = ["1734021", "1734022", "1802256"]
for i in ids:
    print(i, "->", pseudonym(i))
print("same ID, same pseudonym (records can be linked):", pseudonym("1734021") == pseudonym("1734021"))
# A plain unkeyed hash is NOT a safe pseudonym for a small ID space: anyone could
# recompute hashes for every possible ID. The secret key prevents that.
print("ID space size for 7-digit IDs:", 10**7, "-> small enough that unkeyed hashes offer no protection")

1734021 -> dd5e25f97e3114f3
1734022 -> a62033d27f5c40bb
1802256 -> b90655714252925c
same ID, same pseudonym (records can be linked): True
ID space size for 7-digit IDs: 10000000 -> small enough that unkeyed hashes offer no protection

The strongest privacy measure is not to collect data at all. Data minimization asks, for each field, whether the stated purpose truly needs it. An age check needs to know whether someone is over 18, not their birth date; a newsletter needs an email address, not a social insurance number. Data that is never stored cannot be breached, subpoenaed or misused, and minimization reduces the compliance burden at the same time.

from datetime import date
# Data minimization at collection: keep only what the stated purpose needs
signup = {"name": "Ana Lima", "email": "ana@example.org", "birth_date": "2004-03-19",
          "sin": "000-000-000", "phone": "780-555-0100", "program": "Computing Science"}
PURPOSE_FIELDS = {"newsletter": {"email", "program"},
                  "age_check": {"birth_date"}}
def collect(record, purpose):
    kept = {k: v for k, v in record.items() if k in PURPOSE_FIELDS[purpose]}
    dropped = sorted(set(record) - set(kept))
    return kept, dropped
kept, dropped = collect(signup, "newsletter")
print("newsletter keeps:", kept)
print("newsletter drops:", dropped)
# For an age check, store the answer, not the birth date
born = date.fromisoformat(signup["birth_date"])
today = date(2026, 10, 24)
adult = (today.year - born.year - ((today.month, today.day) < (born.month, born.day))) >= 18
print("age check stores only:", {"is_18_or_over": adult})

newsletter keeps: {'email': 'ana@example.org', 'program': 'Computing Science'}
newsletter drops: ['birth_date', 'name', 'phone', 'sin']
age check stores only: {'is_18_or_over': True}

Cryptographic privacy-enhancing technologies

A newer family of techniques lets parties compute on data without seeing it. Homomorphic encryption allows certain computations on encrypted values, producing an encrypted result that only the key holder can decrypt; fully homomorphic schemes can compute anything but remain slow. Secure multi-party computation lets several parties jointly compute a function, such as the average salary across companies, while each learns only the result. Federated learning trains a model across many devices or institutions by sharing model updates rather than raw data, often combined with differential privacy. Trusted execution environments isolate computation inside protected processor enclaves. Zero-knowledge proofs let someone prove a statement, such as being over 18, without revealing the underlying data. These tools are maturing quickly and appear in digital identity wallets, advertising measurement and medical research consortia.

Worked examples