Renyi Differential Privacy Mechanisms for Posterior Sampling

By Joseph Geumlek, Shuang Song, Kamalika Chaudhuri. 2017

64 citations recorded by the source corpus.

Using a recently proposed privacy definition of Renyi Differential Privacy (RDP), we re-examine the inherent privacy of releasing a single sample from a posterior distribution. We exploit the impact of the prior distribution in mitigating the influence of individual data points. In particular, we focus on sampling from an exponential family and specific generalized linear models, such as logistic regression. We propose novel RDP mechanisms as well as offering a new RDP analysis for an existing method in order to add value to the RDP framework. Each method is capable of achieving arbitrary RDP privacy guarantees, and we offer experimental results of their efficacy.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Subsampled Rényi Differential Privacy and Analytical Moments Accountant (2018, methodology): “Our results cover any Rényi diﬀerentially private mechanism, including those based on any exponential family distribution (see Geumlek et al., 2017, and our exposition in Appendix I).”
Differentially-Private "Draw and Discard" Machine Learning (2018, methodology): “…the second – from the similarities between DDML and the shufﬂing strategy of [6] and privacy ampliﬁcation by sampling exploited by [32], [47], [1]; for the third – from recent work in differentially private machine learning [1], [20], [37], [50] that beneﬁts from analyses using such relaxations.”
Automatic Discovery of Privacy–Utility Pareto Fronts (2019, methodology): “This choice is also aligned with recent uses of DP in machine learning where the privacy analysis is conducted under the framework of Rényi DP [39] and the reported privacy is obtained a posteriori by converting the guarantees to standard (ε, δ)-DP for some fixed δ [1, 18, 22, 38, 48].”
Federated Learning with Bayesian Differential Privacy (2019, background): “vide tight composition theorems, along with some other advantages, but to the best of our knowledge, they are not broadly used in practice compared to traditional DP (although there are some examples [29]). One of the possible reasons for that is interpretability: parameters of (;")-RDP or (;˝)”
Submix: Practical Private Prediction for Large-Scale Language Models (2022, methodology): “In particular, S UB M IX uses a privacy accounting methodology based on R´enyi divergences similar to that of Geumlek et al. (2017).”
Differentially Private Bayesian Linear Regression (2019, background): “ sufﬁcient statistic perturbation (SSP) [Foulds et al., 2016, McSherry and Mironov, 2009, Vu and Slavkovic, 2009, Wang, 2018, Zhang et al., 2016], posterior sampling (OPS) [Dimitrakakis et al., 2014, Geumlek et al., 2017, Minami et al., 2016, Wang, 2018, Wang et al., 2015, Zhang et al., 2016], subsa”
Poission Subsampled Rényi Differential Privacy (2019, background): “, the exponential family mechanisms (Geumlek et al., 2017).”
Differentially Private Bayesian Inference for Exponential Families (2018, methodology): “There is a growing interest in private methods for Bayesian inference [10–14].”