Privacy-preserving deep learning
By Reza Shokri, Vitaly Shmatikov. 2015
2,700 citations recorded by the source corpus.
Deep learning based on artificial neural networks is a very popular approach to modeling, classifying, and recognizing complex data such as images, speech, and text. The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based services on the Internet. Commercial companies that collect user data on a large scale have been the main beneficiaries of this trend since the success of deep learning techniques is directly proportional to the amount of data available for training. Massive data collection required for deep learning presents obvious privacy issues. Users' personal, highly sensitive data such as photos and voice recordings is kept indefinitely by the companies that collect it. Users can neither delete it, nor restrict the purposes for which it is used. Furthermore, centrally kept data is subject to legal subpoenas and extrajudicial surveillance. Many data owners-for example, medical institutions that may want to apply deep learning methods to clinical records-are prevented by privacy and confidentiality concerns from sharing the data and thus benefitting from large-scale deep learning. In this paper, we present a practical system that enables multiple parties to jointly learn an accurate neural-network model for a given objective without sharing their input datasets. We exploit the fact that the optimization algorithms used in modern deep learning, namely, those based on stochastic gradient descent, can be parallelized and executed asynchronously. Our system lets participants train independently on their own datasets and selectively share small subsets of their models' key parameters during training. This offers an attractive point in the utility/privacy tradeoff space: participants preserve the privacy of their respective data while still benefitting from other participants' models and thus boosting their learning accuracy beyond what is achievable solely on their own inputs. We demonstrate the accuracy of our privacy-preserving deep learning on benchmark datasets.
Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Health Informatics
- Lingjuan Lyu
- Qiang Yang
- Congzheng Song
- Amir Houmansadr
- Hangyu Zhu
- Yaochu Jin
- David E. Evans
- Bargav Jayaraman
In Safety, Robustness & Interpretability
- Nicholas Carlini
- Nicolas Papernot
- Florian Tramèr
- Reza Shokri
- Dawn Song
- Yang Zhang
- Prateek Mittal
- Vitaly Shmatikov
Citations in the corpus
- Extracting Training Data from Large Language Models (2020, methodology): “An approach to minimizing memorization of training data is to apply differentially-private training of deep learning models [1, 7, 38, 56, 59].”
- Prefix-Tuning: Optimizing Continuous Prompts for Generation (2021, background): “One practical setting is user privacy (Shokri and Shmatikov, 2015; McMahan et al., 2016).”
- Red Teaming Language Models with Language Models (2022, neutral): “LM-based red teaming complements training methods that minimize data leakage, e.g., differential privacy (Chaudhuri and Monteleoni, 2009; Rubinstein et al., 2012; Shokri and Shmatikov, 2015; Abadi et al., 2016).”
- Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing (2017, methodology): “Recently, methods have also been developed to train deep neural networks under differential privacy with formal assurances about privacy risks [19, 20].”
- Towards the Science of Security and Privacy in Machine Learning (2016, background): “Despite noise injected in parameters, Shokri et al. showed that large-capacity models like deep neural networks can be trained with multi-party computations from perturbed parameters and provide differential privacy guarantees [102].”
- Deep Learning with Differential Privacy (2016, methodology): “[52] R. Shokri and V. Shmatikov.”
- Membership Inference Attacks Against Machine Learning Models (2016, background): “Differential privacy [12] has been applied to linear and logistic regression [8], [37], support vector machines [28], risk minimization [5], [9], [34], deep learning [1], [30], learning an unknown probability distribution over a discrete population from random samples [10], and releasing hyper-param”
- How To Backdoor Federated Learning (2018, background): “There exist other flavors of distributed privacy-preserving learning [61], but they are trivial to”
- Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, neutral): “, weights trained in neural networks [21]–[23].”
- Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, methodology): “This poses serious privacy threats, as pointed out by Shokri and Shmatikov in [75], and distrustful”
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, methodology): “Indeed, in our experiments on MNIST and SVHN, the scale of the noise one needs to add to the released ε is smaller than 0.5 and 1.0 respectively.”
- Differentially Private Learning Needs Better Features (or Much More Data) (2020, methodology): “Training these ML models under the framework of differential privacy (DP) (Dwork et al., 2006b; Chaudhuri et al., 2011; Shokri & Shmatikov, 2015; Abadi et al., 2016) can protect deployed classifiers against unintentional leakage of private training data (Shokri et al., 2017; Song et al., 2017;…”
- Privacy in Deep Learning: A Survey (2020, methodology): “w/ DNN Distributed DP [87] # # eICU, TCGA Classification w/ DNN DP model publishing [88] # # MNIST, CIFAR Image Classification w/ DNN DP federated learning [89] # # MNIST Image Classification w/ DNN ScalarDP, PrivUnit [90] # # MNIST, CIFAR Image Classification w/ DNN DSSGD [21] # # MNIST, SVHN Image”
- Communication Efficient Decentralized Training with Multiple Local Updates (2019, background): “Federated Learning (FL), which has a central parameter server, enables massive edge computing devices to jointly learn a centralized model while keeping all local data localized [13, 34, 26, 12, 18, 30, 57].”
- Central Server Free Federated Learning over Single-sided Trust Social Networks (2019, methodology): “Moreover, to prevent reverse engineering of the user’s identity, a certain amount of noise must be added to the gradient to protect user privacy, which partially sacrifices the efficiency and the accuracy (Shokri and Shmatikov, 2015).”
- The Non-IID Data Quagmire of Decentralized Machine Learning (2019, methodology): “As most training data are generated in a decentralized manner, prior work proposes communication-efficient algorithms (e.g., (Hsieh et al., 2017; McMahan et al., 2017; Shokri & Shmatikov, 2015; Lin et al., 2018; Tang et al., 2018)) for ML training over decentralized datasets.”
- AdaCliP: Adaptive Clipping for Private SGD (2019, methodology): “Since [13] clips each gradient entry and [14] clips the entire gradient norm, for same clip thresholds, [13] incurs much more privacy loss compared to [14].”
- Learning Privately over Distributed Features: An ADMM Sharing Approach (2019, background): “elated Work Machine Learning Algorithms and Privacy. Chaudhuriand Monteleoni (2009) is one of the first studies combing machine learning and differential privacy (DP), focusing on logistic regression. Shokri and Shmatikov (2015) applies a variant of SGD to collaborative deep learning in a dataparalle”