On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis

By James R. Foulds, Joseph Geumlek, Max Welling, Kamalika Chaudhuri. 2016

104 citations recorded by the source corpus.

Bayesian inference has great promise for the privacy-preserving analysis of sensitive data, as posterior sampling automatically preserves differential privacy, an algorithmic notion of data privacy, under certain conditions (Dimitrakakis et al., 2014; Wang et al., 2015). While this one posterior sample (OPS) approach elegantly provides privacy "for free," it is data inefficient in the sense of asymptotic relative efficiency (ARE). We show that a simple alternative based on the Laplace mechanism, the workhorse of differential privacy, is as asymptotically efficient as non-private posterior inference, under general assumptions. This technique also has practical advantages including efficient use of the privacy budget for MCMC. We demonstrate the practicality of our approach on a time-series analysis of sensitive military records from the Afghanistan and Iraq wars disclosed by the Wikileaks organization.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Subsampled Rényi Differential Privacy and Analytical Moments Accountant (2018, methodology): “A popular way of designing differentially private machine learning models is to use Stochastic Gradient Descent (SGD) with differentially private releases of (sometimes clipped) gradients evaluated on mini-batches of a dataset (Song et al., 2013; Bassily et al., 2014; Wang et al., 2015; Foulds et al”
Poission Subsampled Rényi Differential Privacy (2019, methodology): “One prominent use case of both techniques is the NoisySGD algorithm (Song et al., 2013; Bassily et al., 2014; Wang et al., 2015; Foulds et al., 2016; Abadi et al., 1UC Santa Barbara, Department of Computer Science.”
Renyi Differential Privacy Mechanisms for Posterior Sampling (2017, methodology): “As a baseline, we select a modified version of the algorithm in [10], which privatizes the sufficient statistic of the data to create a privatized posterior.”
Per-instance Differential Privacy (2017, methodology): “Secondly, our bound has explicit dimension dependence while (Foulds et al., 2016) and (Smith, 2008) treat d as a constant.”
Differentially Private Bayesian Inference for Exponential Families (2018, methodology): “We will adopt a straightforward release mechanism where the Laplace mechanism [1] is used to release noisy sufﬁcient statistics y [12, 19], which are a ﬁnite-dimensional quantity that capture all the information about θ [21].”
Per-instance Differential Privacy and the Adaptivity of Posterior Sampling in Linear and Ridge regression (2017, methodology): “This proposition reveals that AdaOPS improves over previous results in the literature [28, 17] in several ways.”
Profile-based Privacy for Locally Private Computations (2019, background): “A body of compelling results [4, 5, 13, 10, 18] have been achieved in the ”centralized” model, in which a trusted data curator has raw access to the data while performing the privacy-preserving operations.”
Differentially Private Bayesian Linear Regression (2019, methodology): “A variety of privacy mechanisms have been applied to point estimation of regression coefﬁcients, including sufﬁcient statistic perturbation (SSP) [Foulds et al., 2016, McSherry and Mironov, 2009, Vu and Slavkovic, 2009, Wang, 2018, Zhang et al., 2016], posterior sampling (OPS) [Dimitrakakis et al.,…”