Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

By Nicolas Papernot, Martín Abadi, Ú. Erlingsson, Ian J. Goodfellow, Kunal Talwar. International Conference on Learning Representations, 2016

1,156 citations recorded by the source corpus.

Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information. To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data: Private Aggregation of Teacher Ensembles (PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student's privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student's training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings. Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

View Paper: https://arxiv.org/abs/1610.05755

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, methodology): “We use the PATE framework for learning with privacy.”
Towards the Science of Security and Privacy in Machine Learning (2016, neutral): “This strategy was explored in [98], [99].”
How To Backdoor Federated Learning (2018, background): “rential privacy. Differentially private federated learning [24] bounds each participant’s inﬂuence over the joint model. In Section 6.3, we evaluate the extent to which it mitigates our attacks. PATE [28, 30] uses knowledge distillation [16] to transfer knowledge from “teacher” models trained on pri”
Differentially Private Learning Needs Better Features (or Much More Data) (2020, background): “We focus on CIFAR-10, and discuss two natural paths towards better private models: (1) access to a larger private training set, and (2) access to a public image dataset from a different distribution (some works also consider access to public unlabeled data from the same distribution as the private d”
Training Production Language Models without Memorizing User Data (2020, methodology): “Many prior works in DP machine learning (ML) [CMS11], [BST14], [ACG + 16], [PAE + 16], [WLK + 17], [PSM + 18], [INS + 19] deal with example-level privacy, i.e., providing privacy guarantees for any single example in a dataset.”
NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, background): “The works in [45, 46, 47] are examples of such schemes for transfer learning, subspace clustering.”
Privacy in Deep Learning: A Survey (2020, methodology): “, 2017b) # # MNIST, CIFAR-10 Image Classification w/ DNN PATE(Papernot et al., 2016) # # MNIST, SVHN Image Classification w/ DNN Scalable Learning w/ PATE(Papernot et al.”
Generative Models for Effective ML on Private, Decentralized Datasets (2019, background): “Most prior work on differentially private ML (e.g. Chaudhuri et al. (2011); Bassily et al. (2014); Abadi et al. (2016); Wu et al. (2017); Papernot et al. (2017)) deals with example-level privacy: two datasets d and d′ are defined to be adjacent if d′ can be formed by adding or removing a single…”
Secure Evaluation of Quantized Neural Networks (2019, neutral): “[44]) and can be seen as complimentary to this work.”
Differentially Private Learning with Adaptive Clipping (2019, neutral): “In example-level DP, datasets are considered neighbors when they differ by the addition or removal of a single example (Chaudhuri et al., 2011; Bassily et al., 2014; Abadi et al., 2016b; Papernot et al., 2017; Wu et al., 2017; Papernot et al., 2018; Iyengar et al., 2019).”
A General Approach to Adding Differential Privacy to Iterative Training Procedures (2018, neutral): “…on integrating differential privacy (DP) techniques into iterative training procedures like stochastic gradient descent [Chaudhuri et al., 2011, Bassily et al., 2014, Abadi et al., 2016, Wu et al., 2017, Papernot et al., 2017]; for completeness we provide a formal definition of DP in Appendix A.”
The Power of the Hybrid Model for Mean Estimation (2018, background): “In this model, some works assume that DP is achieved in the TCM [35, 50], while others assume that DP is achieved in the LM [55, 58].”
Privacy Amplification by Iteration (2018, background): “: The setting in which some public data from the same distribution as private data is available has been recently identified as promising and practically important [10], [11].”
Distributed learning of deep neural network over multiple agents (2018, background): “In future work, a learned neural network could be shared using student-teacher methods for transferring information learned by neural network [41].”
Differentially-Private "Draw and Discard" Machine Learning (2018, background): “ML in the Trusted-Curator Model: Most prior work for differentially private machine learning assumes a trusted-curator model, where the data is ﬁrst collected by the company and only then a privacy-preserving computation is run on it [1], [48], [42], [9].”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “Transfer learning in combination with differentially private (DP) techniques tailored for deep learning [1] has been used in [45, 46].”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “Such sample-level privacy can be achieved in many learning tasks [2, 7, 22, 53, 85, 86].”