Rényi Differential Privacy
By Ilya Mironov. 2017
1,571 citations recorded by the source corpus.
We propose a natural relaxation of differential privacy based on the Rényi divergence. Closely related notions have appeared in several recent papers that analyzed composition of differentially private mechanisms. We argue that the useful analytical tool can be used as a privacy definition, compactly and accurately representing guarantees on the tails of the privacy loss.We demonstrate that the new definition shares many important properties with the standard definition of differential privacy, while additionally allowing tighter analysis of composite heterogeneous mechanisms.
Subfields: Artificial Intelligence, Sociology and Political Science, Privacy & Security
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Nicholas Carlini
- Dawn Song
- Philip S. Yu
- Nicolas Papernot
- Peter Kairouz
- Florian Tramèr
- Yoshua Bengio
In Sociology and Political Science
- Daniel Kifer
- Abhradeep Thakurta
- Yu-Xiang Wang
- Kamalika Chaudhuri
- Vitaly Feldman
- Kunal Talwar
- Borja Balle
- Shuang Song
In Privacy & Security
- Abhradeep Thakurta
- Peter Kairouz
- Borja Balle
- Vitaly Feldman
- Kunal Talwar
- Yu-Xiang Wang
- H. Brendan McMahan
- Jonathan Ullman
Citations in the corpus
- Deep Learning with Differential Privacy (2016, background): “The moments accountant is closely related to the notion of Rényi differential privacy [44], which proposes (scaled) α(λ) as a means of quantifying privacy guarantees.”
- Advances and Open Problems in Federated Learning (2019, methodology): “To track the overall privacy budget across rounds, advanced composition theorems [148, 221] or the analytical moments accountant method developed in [7, 297, 299, 405] can be used.”
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, methodology): “The moments accountant was introduced by Abadi et al. (2016), building on previous work (Bun & Steinke, 2016; Dwork & Rothblum, 2016; Mironov, 2016).”
- Practical and Private (Deep) Learning without Sampling or Shuffling (2021, background): “, 2006b; Dwork & Roth, 2014) and (α, α/2σ(2))-RDP (Mironov, 2017).”
- The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation (2021, methodology): “We now state the definitions of concentrated DP (Bun & Steinke, 2016) and Rényi DP (Mironov, 2017) and relate these to the standard definition of differential privacy (Dwork et al., 2006b;a).”
- DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “These mechanisms are sometimes built under the framework of differential privacy [16] or its derivatives [64, 65].”
- Privacy Amplification by Decentralization (2020, methodology): “Finally, we can translate the result of the RDP by using the following result (Mironov, 2017).”
- Differentially Private Learning Needs Better Features (or Much More Data) (2020, methodology): “When applying Data Normalization to ScatterNet features, we compute the per-channel statistics only over the original CIFAR-10 samples, and compute the privacy guarantees of PrivDataNorm using the Rényi DP analysis of the sampled Gaussian mechanism (Mironov et al., 2019; Wang et al., 2019).”
- IBM Federated Learning: an Enterprise Framework White Paper V0.1 (2020, methodology): “Multiple variations on differential privacy methods may be implemented including [5, 22].”
- Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy (2020, background): “Moreover, new DP mechanisms such as RAPPOR [46] will be added, together with relaxations of DP such as Concentrated DP [47] or RÃl’nyi DP [48].”
- Generative Models for Effective ML on Private, Decentralized Datasets (2019, methodology): “using the strong composition theorem for DP (Mironov, 2017; Kairouz et al., 2017).”
- Differential Privacy Has Disparate Impact on Model Accuracy (2019, methodology): “Given T training epochs, dataset size N , batch size b , noise multiplier z , and δ , we compute privacy loss for each training run using the Rényi DP [28] implementation from TF Privacy [36].”
- Differentially Private Learning with Adaptive Clipping (2019, neutral): “Standard composition techniques can then be used to extend the per-update guarantee to the final model [20].”
- Protection Against Reconstruction and Its Applications in Private Federated Learning (2018, background): “Mironov [59] shows that if M is ε-differentially private, then for any α ≥ 1, it is (min{ε, 2αε2}, α)Rényi private, and conversely, if M is (ε, α)-Rényi private, it also satisfies (ε+ log(1/δ) α−1 , δ)-differential privacy for all δ ∈ [0, 1].”
- Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity (2018, background): “5 64 n"2 1 65"2 0 n 2 3 " 0 r log(1=) n : Combining these terms gives the claimed bound. Remark 1. Our bound can also be easily stated in terms of Renyi differential privacy (RDP) [Mir17]. Using the fact that "-DP implies (;"2=2)-RDP [BS16] and the composition properties o”