Deep Learning with Differential Privacy
By Martín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang. 2016
8,077 citations recorded by the source corpus.
Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains. Often, the training of models requires large, representative datasets, which may be crowdsourced and contain sensitive information. The models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Jianfeng Gao
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
- Sergey Levine
In Safety, Robustness & Interpretability
- Nicholas Carlini
- Nicolas Papernot
- Florian Tramèr
- Dawn Song
- Reza Shokri
- Prateek Mittal
- Bo Li
- Vitaly Shmatikov
Citations in the corpus
- GLaM: Efficient Scaling of Language Models with Mixture-of-Experts (2021, methodology): “…et al., 2021), including bias in representations (Blodgett et al., 2020), proper handling (Rogers, 2021) and documentation (Bender & Friedman, 2018) of training data, privacy (Abadi et al., 2016b; Carlini et al., 2021), and environmental concerns (Strubell et al., 2019; Patterson et al., 2021).”
- Improving language models by retrieving from trillions of tokens (2021, background): “In addition, differential privacy training (Abadi et al., 2016) of retrieval models could guarantee that no private information is stored in the model weights, while individualisation on private data could be made by updating the retrieval database at inference time.”
- Quantifying Memorization Across Neural Language Models (2022, methodology): “Because existing approaches for provably-correct privacy-preserving training of machine learning are applied only rarely in practice (Abadi et al., 2016; Thakkar et al., 2020; Ramaswamy et al., 2020), it is common to attempt post-hoc privacy auditing (Jayaraman and Evans, 2019; Jagielski et al.”
- Extracting Training Data from Large Language Models (2020, methodology): “Private machine learning models can be trained with variants of the differentially private stochastic gradient descent (DP-SGD) algorithm [1] which is widely implemented [15, 22].”
- MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, methodology): “Hence, approaches for deep learning from private training data, such as DP-SGD [1] and PATE [36], benefit from accessing as few labeled private training points as possible when computing updates to the model parameters.”
- Ethical Challenges in Data-Driven Dialogue Systems (2017, background): “Future lines of investigation may find the integration of automated anonymization [29] and differential privacy [1] beneficial for conversational models.”
- Improving alignment of dialogue agents via targeted human judgements (2022, background): “Issues such as privacy (Abadi et al., 2016) and social, linguistic or environmental justice (Bender et al., 2021) require mitigations at pretraining time in addition to downstream work, though rules have a role (such as teaching an agent to not reveal information that should be private, even if it…”
- Red Teaming Language Models with Language Models (2022, neutral): “LM-based red teaming complements training methods that minimize data leakage, e.g., differential privacy (Chaudhuri and Monteleoni, 2009; Rubinstein et al., 2012; Shokri and Shmatikov, 2015; Abadi et al., 2016).”
- Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing (2017, methodology): “[19] to keep track of the privacy parameters (ε, δ) over time.”
- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, methodology): “We applied the differentially-private stochastic gradient descent algorithm (DP-SGD) from [1] to verify that differential privacy is an effective defense that prevents memorization.”
- Towards the Science of Security and Privacy in Machine Learning (2016, methodology): “Later, Abadi et al. introduced an alternative approach to improve the privacy guarantees provided: the strategy followed is to randomly perturb parameters during the stochastic gradient descent performed by the learning algorithm [103].”
- Membership Inference Attacks Against Machine Learning Models (2016, background): “Differential privacy [12] has been applied to linear and logistic regression [8], [37], support vector machines [28], risk minimization [5], [9], [34], deep learning [1], [30], learning an unknown probability distribution over a discrete population from random samples [10], and releasing hyper-param”
- Unsolved Problems in ML Safety (2021, background): “There is a large ecosystem of work on addressing societal consequences of machine learning, including AI policy [43], privacy [3, 162], fairness [72], and ethics [56].”
- Advances and Open Problems in Federated Learning (2019, methodology): “To track the overall privacy budget across rounds, advanced composition theorems [148, 221] or the analytical moments accountant method developed in [7, 297, 299, 405] can be used.”
- How To Backdoor Federated Learning (2018, background): “Recent work [20, 44] showed how to use federated learning for word prediction with participant-level differential privacy [1].”
- Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, methodology): “We can note that parameter clipping C is a popular ingredient of SGD and ML for non-privacy reasons.”