Differential Privacy as a Mutual Information Constraint

By Paul W. Cuff, Lanqing Yu. 2016

229 citations recorded by the source corpus.

Differential privacy is a precise mathematical constraint meant to ensure privacy of individual pieces of information in a database even while queries are being answered about the aggregate. Intuitively, one must come to terms with what differential privacy does and does not guarantee. For example, the definition prevents a strong adversary who knows all but one entry in the database from further inferring about the last one. This strong adversary assumption can be overlooked, resulting in misinterpretation of the privacy guarantee of differential privacy. Herein we give an equivalent definition of privacy using mutual information that makes plain some of the subtleties of differential privacy. The mutual-information differential privacy is in fact sandwiched between ε-differential privacy and (ε,δ)-differential privacy in terms of its strength. In contrast to previous works using unconditional mutual information, differential privacy is fundamentally related to conditional mutual information, accompanied by a maximization over the database distribution. The conceptual advantage of using mutual information, aside from yielding a simpler and more intuitive definition of differential privacy, is that its properties are well understood. Several properties of differential privacy are easily verified for the mutual information alternative, such as composition theorems.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Deep Variational Information Bottleneck (2017, methodology): “…real images; using richer parametric marginal approximations, rather than assuming r ( z ) = N (0 , I ) ; exploring the connections to differential privacy (see e.g., Wang et al. (2016a); Cuff & Yu (2016)); and investigating open universe classiﬁcation problems (see e.g., Bendale & Boult (2015)).”
A Principled Approach to Learning Stochastic Representations for Privacy in Deep Neural Inference (2020, methodology): “The amount of mutual information between the raw data and the representation that is to be publicized is another measure of privacy that is widely used in literature (Cuff & Yu, 2016; Kalantari et al., 2017; Liao et al., 2017).”
Obfuscation via Information Density Estimation (2019, background): “In fact, the information density appears in DP under the name of privacy loss variable (cf., e.g., Dwork and Rothblum (2016), thereby connecting DP and information-theoretic quantities, e.g., mutual information DP (Cuff and Yu, 2016) and Rényi DP (Mironov, 2017).”
Differentially Private Testing of Identity and Closeness of Discrete Distributions (2017, background): “Information theoretic approaches to data privacy have also been studied in the past few years using quantities like mutual information, and guessing probability to quantify privacy [Mir12, SRP13, CY16, WYZ16, IW17].”