On the Difficulties of Disclosure Prevention in Statistical Databases or The Case for Differential Privacy

By Cynthia Dwork, Moni Naor. 2010

154 citations recorded by the source corpus.

In 1977 Tore Dalenius articulated a desideratum for statistical databases: nothing about an individual should be learnable from the database that cannot be learned without access to the database. We give a general impossibility result showing that a natural formalization of Dalenius’ goal cannot be achieved if the database is useful. The key obstacle is the side information that may be available to an adversary. Our results hold under very general conditions regarding the database, the notion of privacy violation, and the notion of utility. Contrary to intuition, a variant of the result threatens the privacy even of someone not in the database. This state of affairs motivated the notion of differential privacy [15, 16], a strong ad omnia privacy which, intuitively, captures the increased risk to one’s privacy incurred by participating in a database.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Privacy & Security

Citations in the corpus

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “In general, privacy cannot be guaranteed if auxiliary information (outside the DP model) is accessible to the adversary [21].”
Membership Inference Attacks Against Machine Learning Models (2016, background): “It is not possible to prevent disclosure due to population statistics [14].”
Differential Privacy for Protecting Multi-dimensional Contingency Table Data: Extensions and Applications (2012, unknown): “v∅ = [ 1 8 , 1 8 , 1 8 , 1 8 , 1 8 , 1 8 , 1 8 , 1 8 , 1 8 ]T ,”
Inference Attacks Against Collaborative Learning (2018, background): “If a machine learning (ML) model is useful, it must reveal information about the data on which it was trained [13].”
Pufferfish (2014, background): “…Privacy Deﬁnitions 3:5 Note that assumptions are absolutely necessary—privacy deﬁnitions that can provide privacy guarantees without making any assumptions provide little utility beyond the default approach of releasing nothing at all [Kifer and Machanavajjhala 2011; Dwork and Naor 2010].”
Coupled-Worlds Privacy: Exploiting Adversarial Uncertainty in Statistical Data Privacy (2013, background): “• Analyzing existing mechanisms: A broader goal is to understand what privacy guarantees are achieved by methods in use today (e.g., disclosure-control methods currently used by statistical agencies, or releases that are mandated by law) that were not designed with specific privacy definitions in…”
Is privacy privacy? (2018, methodology): “in data analysis would essentially prohibit the discovery of any new knowledge through research using personal data [29].”
On the 'Semantics' of Differential Privacy: A Bayesian Formulation (2008, background): “The schemes in all those papers, however, satisfy the following relaxation (first formulated by Dwork, Kenthapadi, McSherry, Mironov, and Naor [6]): Definition 1.2 (( , δ)-differential privacy [6]).”
No free lunch in data privacy (2011, methodology): “In particular our contributions are that we (1) simplify a result of Dwork and Naor [8, 11] in order to show how privacy relies on assumptions about data generation, (2) propose a participation-based guideline does deleting an individual s tuple erase all evidence of that individual s participa”
Membership Inference Attacks on Machine Learning: A Survey (2021, background): “[21] argue that if a machine learning model is useful, it must reveal information about the data on which it was trained.”
A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “• The inference of memberships aims to determine whether a data sample has been used for model training [42, 114, 123, 129, 130, 139, 160].”
A Review on Fairness in Machine Learning (2022, background): “, as captured by the term inferential privacy [53, 64, 82]) can lead to unfairness due to the ability of an adversary to infer sensitive information about an individual and use it in a discriminating manner.”
Distributed Differential Privacy and Applications (2015, background): “Instead of attempting to scrub a dataset until it is “perfectly anonymized”, an impossible target (Dwork and Naor, 2008), we could try a second approach: releasing just the result of a computation based on the private data, without releasing the dataset itself.”
Differential Privacy for Statistics: What we Know and What we Want to Learn (2010, methodology): “The heart of the attack uses extracted randomness from the statistical database as a one-time pad for conveying the privacy compromise to the adversary/user [14, 20].”