Big question: Can information be non-secret and still used inappropriately?
Research lock: 2026-08-28
Why this week matters
AI shifts privacy from a question of hidden records to a question of inference and capability. A few ordinary facts can produce a sensitive prediction; public photos can become biometric search; aggregate statistics can expose individuals; deletion of a record may not remove embeddings, templates, model influence, or downstream copies. Privacy therefore concerns appropriate information flow, power, and contestability—not secrecy alone.
Deeper teaching spine
1. Model an information flow
Use Nissenbaum's contextual-integrity parameters: data subject, sender, recipient, information type, and transmission principle. Add purpose, retention, derived data, downstream recipient, and remedy for an AI system. A flow can violate a norm even when every individual datum was observable.
Example: a person posts a conference photo for colleagues. The same image is scraped, converted into a biometric template, linked to an identity, and offered to police as a search capability. Access remained public; actors, purpose, attributes, and scale changed.
2. Distinguish privacy attacks and governance failures
- Re-identification: connect supposedly anonymous records to a person.
- Attribute inference: infer a sensitive property not directly supplied.
- Membership inference: estimate whether a record was in training data.
- Model inversion/extraction: recover information or behavior from a model.
- Purpose expansion: reuse data outside the context that justified collection.
- Aggregation: turn many low-sensitivity observations into a high-power dossier.
The technical attack matters, but so do retention, access control, procurement, audit logging, contracts, and recourse.
3. Differential privacy is a bounded promise
Explain neighbouring datasets: a mechanism is differentially private when its output distribution changes only within a bounded amount if one person's data is added or removed. Epsilon encodes a privacy-loss budget; it is not a universal “privacy score.” Composition, repeated releases, population size, query design, and utility all matter.
The 2020 Census case lets students see distributional stakes. Noise protects respondents but can affect small-area counts and downstream resource allocation. Ask who chooses the privacy–utility trade-off, who bears error, and how uncertainty should be communicated.
4. Automated decisions require information rights
The EDPB guidance and the CJEU's SCHUFA judgment make automated scoring more than a disclosure problem. Students should ask what qualifies as a decision based solely or substantially on automation, what meaningful information is needed, and what contestation can change. A generic model description may be transparent yet useless to an affected person.
5. Design deletion around a data lineage
Create a lineage from raw record to cleaned row, feature, embedding, biometric template, dataset snapshot, trained checkpoint, evaluation log, cache, vendor copy, and downstream decision. At each node record owner, retention, legal basis, deletion method, and verification. If deletion cannot reach a node, say so rather than promising a fiction.
Case-study dossier
Clearview AI: public images, transformed context
Use the Canadian privacy regulators' joint findings as the anchor. Ask which transformation created the privacy interest: scraping, biometric templating, identity linkage, search, disclosure, or scale. Then design a deletion order that can be verified.
Census differential privacy: privacy has distributional effects
Assign students to households, small municipalities, Census Bureau statisticians, researchers, and funding agencies. Each proposes an acceptable release and evidence plan. The point is not to solve epsilon in class; it is to make governance choices around a formal guarantee visible.
SCHUFA: when a score effectively determines a decision
The CJEU's December 2023 judgment treated scoring as automated decision-making where clients gave it a determining role. Compare a score that informs a human with one that the downstream institution routinely follows. “Human in the loop” is not meaningful if the human lacks time, authority, information, or a real alternative.
Seminar activities
- Context audit: map one familiar data flow—learning analytics, health app, workplace monitoring, or loyalty program—and circle every changed actor, purpose, or transmission principle.
- Inference surprise: provide innocuous features and ask teams to list sensitive attributes or behaviours that could be inferred. Then prohibit one feature and see whether proxies remain.
- Deletion proof: draft evidence that a vendor must provide after a deletion request, including derivatives and subprocessors.
- Privacy budget hearing: decide whether an additional statistical release is worth consuming a finite privacy budget and who benefits.
Visual evidence plan
| Visual | Capture target | Teaching use | Guardrail |
|---|---|---|---|
| Clearview findings | public/courses/mai-105/evidence/clearview-opc.jpg | Mark the chain from public photo to biometric search. | Do not equate access with consent. |
| Census disclosure-avoidance page | U.S. Census Bureau | Capture the Bureau's own explanation of why and how protection was used. | Pair with stakeholder criticism; an agency page is not neutral evaluation. |
| Contextual integrity matrix | Nissenbaum article | Turn actors, attributes, and transmission principles into a flow diagram. | It is a normative framework, not a legal test everywhere. |
| SCHUFA judgment receipt | CJEU press release PDF | Crop the finding that scoring may be automated decision-making when decisive in practice. | Use the judgment/press release, not a vendor summary. |
| Automated-decision guidance | EDPB guidelines | Capture the decision tree for Article 22 discussion. | Explain territorial/legal scope. |
Reading and citation ledger
- Helen Nissenbaum, “Privacy as Contextual Integrity”.
- Office of the Privacy Commissioner of Canada, Clearview AI joint investigation.
- U.S. Census Bureau, Disclosure Avoidance and the 2020 Census.
- EDPB, Guidelines on Automated Individual Decision-Making and Profiling.
- Court of Justice of the European Union, SCHUFA judgment press release (2023).
- FTC, Everalbum facial-recognition settlement — a concrete derivative-deletion remedy.
Watch list
- Privacy law is jurisdiction-specific; label GDPR, Canadian, U.S., and institutional policy claims separately.
- Never describe differential privacy without epsilon, composition, release context, and utility consequences.
- A model may retain influence without allowing literal recovery of a record; separate privacy risk, provenance, and deletion-verification claims.