A General Approach to Adding Differential Privacy to Iterative Training Procedures

By H. Brendan McMahan, Galen Andrew. 2018

216 citations recorded by the source corpus.

In this work we address the practical challenges of training machine learning models on privacy-sensitive datasets by introducing a modular approach that minimizes changes to training algorithms, provides a variety of configuration strategies for the privacy mechanism, and then isolates and simplifies the critical logic that computes the final privacy guarantees. A key challenge is that training algorithms often require estimating many different quantities (vectors) from the same set of examples --- for example, gradients of different layers in a deep learning architecture, as well as metrics and batch normalization parameters. Each of these may have different properties like dimensionality, magnitude, and tolerance to noise. By extending previous work on the Moments Accountant for the subsampled Gaussian mechanism, we can provide privacy for such heterogeneous sets of vectors, while also structuring the approach to minimize software engineering challenges.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, methodology): “Sources of randomness (adapted from [288]) Most computational devices have access only to few sources of entropy and they tend to be very low rate (hardware interrupts, on-board sensors).”
Practical and Private (Deep) Learning without Sampling or Shuffling (2021, background): “, 2016) has become state-of-the-art in training private (deep) learning models (Abadi et al., 2016; McMahan et al., 2018; Erlingsson et al., 2020; Papernot et al., 2020b; Facebook, 2020; Tramèr & Boneh, 2021).”
Tempered Sigmoid Activations for Deep Learning with Differential Privacy (2020, methodology): “Unlike prior work, which attempted to adapt the clipping norm to the gradients of each layer’s parameters post hoc to training (McMahan et al. 2018), we find that tempered sigmoids preserve more of the signal contained in gradients of each layer because they rescale each layer’s activations and bett”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, methodology): “DP-SGD has enabled the development of deep image classiﬁcation and language models [1, 24, 26, 36] that achieve DP with (cid:15) in the single digits at the cost of a modest reduction in the model’s test accuracy.”
Differentially Private Learning with Adaptive Clipping (2019, positive): “…reduces the variance in the updates to a level where we can reasonably assume that adding more clients is unlikely to change convergence properties (McMahan et al., 2018), giving us conﬁdence that our results on the smaller datasets would still hold if it were possible scale up the number of…”
Automatic Discovery of Privacy–Utility Pareto Fronts (2019, methodology): “izing, but privacydependent, hyperparameters. This is particularly challenging, since the privacy’s dependence on the hyperparameters may be non-analytical and computationally expensive to determine. [34, 43] provide approaches to this variant, however the proposed strategies are 1) based on heurist”
Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference (2019, methodology): “We used the Tensorflow Privacy library [29], an implementation of the moments accountant method [1], which guarantees (ε,δ)-differential privacy, to study the practical efficacy of our attack on protected models.”
Federating Recommendations Using Differentially Private Prototypes (2020, background): “…to impose in iterative algorithms, such as those commonly used in federated learning scenarios, since the iterative nature of these algorithms requires splitting privacy budget across iterations, thus bringing forth technical challenges Abadi et al. (2016); Wu et al. (2017); McMahan et al. (2018).”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “Algorithm 3 DP-SGD with microbatching (McMahan & Andrew, 2018) with a minor correction to account for 2C sensitivity Input: Training data, consisting of features X := {x1, x2, ..., xN} and labels Y := {y1, y2, ..., yN}. f(x; θ) is the output of a model parameterized by θ and applied to an input x.”
Deep Learning with Gaussian Differential Privacy (2019, unknown): “This technique has been extended to a variety of situations by follow-up work [39, 46].”
Differentially Private Learning with Adaptive Clipping (2019, methodology): “We also define a parameter c ∈ [0, 1] denoting the proportion of the per-iteration privacy budget (details in [18]) that is used for the computation of the clipped counts described above.”
Shifting machine learning for healthcare from development to deployment and from models to data (2022, background): “Differential privacy 84 can further reinforce privacy protection for federated learning 70,85,86 .”
Unlocking High-Accuracy Differentially Private Image Classification through Scale (2022, methodology): “First, we revisit ideas that have previously been identiﬁed as useful for private training, including using large batch sizes (McMahan et al., 2018b), and replacing batch normalization layers with alternatives that ensure good signal propagation at initialization (van der Maaten and Hannun, 2020).”
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (2022, background): “Instead, pre-training papers have focused their attention on alternatives to ﬁltering, like deduplication [62], federated learning [108, 50], differential privacy [80, 42], and other approaches [32, 74, 65, 127, 82].”
A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences (2020, background): “, the implementation of [9] in the Google’s open-source TensorFlow Privacy framework [10].”
Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020, methodology): “To provide strong privacy guarantees where technologies are built upon potentially sensitive training data, Google recently released TensorFlow Privacy55 [168], a toolkit for TensorFlow that implements techniques based on differential privacy.”