A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences

By Shahab Asoodeh, Jiachun Liao, Flávio du Pin Calmon, Oliver Kosut, Lalitha Sankar. 2020

42 citations recorded by the source corpus.

We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Renyi´ differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Renyi variations of differential privacy. We apply our result to´ the moments accountant framework for characterizing privacy guarantees of stochastic gradient descent. When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Practical and Private (Deep) Learning without Sampling or Shuffling (2021, positive): “Follow-up works (Asoodeh et al., 2020; Canonne et al., 2020) provide tighter conversions.”
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation (2021, background): “In addition we have the following conversion lemma (Bun & Steinke, 2016; Canonne et al., 2020; Asoodeh et al., 2020) from concentrated DP to approximate DP.”
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning (2021, background): “Since then, many works [13, 2, 54, 41, 31, 3] improved the analysis of the Moments Accountant analysis to get better theoretical privacy bounds.”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, background): “Rényi (O(q(es − 1)T ), δ), k slightly less than 1/2 Conversion from RDP to (ε, δ) via Lemma 1 (Asoodeh et al., 2020).”