Scalable Private Learning with PATE

By Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Ú. Erlingsson. 2018

725 citations recorded by the source corpus.

The rapid adoption of machine learning has increased concerns about the privacy implications of machine learning models trained on sensitive data, such as medical records or other personal information. To address those concerns, one promising approach is Private Aggregation of Teacher Ensembles, or PATE, which transfers to a "student" model the knowledge of an ensemble of "teacher" models, with intuitive privacy provided by training teachers on disjoint data and strong privacy guaranteed by noisy aggregation of teachers' answers. However, PATE has so far been evaluated only on simple classification tasks like MNIST, leaving unclear its utility when applied to larger-scale learning tasks and real-world datasets. In this work, we show how PATE can scale to learning tasks with large numbers of output classes and uncurated, imbalanced training data with errors. For this, we introduce new noisy aggregation mechanisms for teacher ensembles that are more selective and add less noise, and prove their tighter differential-privacy guarantees. Our new mechanisms build on two insights: the chance of teacher consensus is increased by using more concentrated noise and, lacking consensus, no answer need be given to a student. The consensus answers used are more likely to be correct, offer better intuitive privacy, and incur lower-differential privacy cost. Our evaluation shows our mechanisms improve on the original PATE on all measures, and scale to larger tasks with both high utility and very strong privacy ($\varepsilon$ < 1.0).

Subfields: Artificial Intelligence, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Privacy & Security

Citations in the corpus

MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, neutral): “96 [35].”
How To Backdoor Federated Learning (2018, background): “PATE [32], [34] uses knowledge distillation [16] to transfer knowledge from “teacher” models trained on private data to a “student” model.”
Differentially Private Learning Needs Better Features (or Much More Data) (2020, methodology): “…better private models: (1) access to a larger private training set, and (2) access to a public image dataset from a different distribution (some works also consider access to public unlabeled data from the same distribution as the private data (Papernot et al., 2017; 2018; Zhu et al., 2020)).”
Training Production Language Models without Memorizing User Data (2020, background): “Many prior works in DP machine learning (ML) [CMS11], [BST14], [ACG + 16], [PAE + 16], [WLK + 17], [PSM + 18], [INS + 19] deal with example-level privacy, i.e., providing privacy guarantees for any single example in a dataset.”
Auditing Differentially Private Machine Learning: How Private is Private SGD? (2020, background): “Diﬀerential privacy [DMNS06] has become the de facto standard for guaranteeing privacy in machine learning and statistical analysis, and is now being deployed by many organizations including Apple [TVV + 17], Google [EPK14, BEM + 17, PSM + 18], and the US Census Bureau [HMA + 17].”
Privacy in Deep Learning: A Survey (2020, methodology): “use MA to aid bounding the privacy budget for their teacher ensemble method that uses noisy voting and label perturbation [23, 85].”
Differentially Private Learning with Adaptive Clipping (2019, neutral): “In example-level DP, datasets are considered neighbors when they differ by the addition or removal of a single example (Chaudhuri et al., 2011; Bassily et al., 2014; Abadi et al., 2016b; Papernot et al., 2017; Wu et al., 2017; Papernot et al., 2018; Iyengar et al., 2019).”
Privacy Amplification by Iteration (2018, background): “This approach improves on traditional (ε, δ)-DP accounting in numerous settings, often leading to significantly tighter privacy bounds as well as being applicable when the traditional approach fails [10], [36].”
Differentially-Private "Draw and Discard" Machine Learning (2018, methodology): “Since differential privacy started to gain traction in industry, there has been signiﬁcant debate devoted to establishing what is a reasonable privacy loss rate and to optimizing the analyses and algorithms to reduce the privacy loss needed [25], [49], [43].”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “Transfer learning in combination with differentially private (DP) techniques tailored for deep learning [1] has been used in [45, 46].”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “Such sample-level privacy can be achieved in many learning tasks [2, 7, 22, 53, 85, 86].”
Comprehensive Privacy Analysis of Deep Learning: Stand-alone and Federated Learning under Passive and Active White-box Inference Attacks (2018, background): “Differential privacy is a strong defense methods against inference attacks [30], [31], which is applied in the context of machine learning [32]–[35].”
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer (2019, methodology): “This is the main objective of knowledge transfer [21], [9], [43], [44], [54], [7], which is a powerful tool to share knowledge of a model through its predictions.”
A Hybrid Approach to Privacy-Preserving Federated Learning (2018, methodology): “Models produced from our FL system learn from all available data, leading to more accurate models than the local models trained by each participant in PATE (Figure 4b in [32] demonstrates the need for a lot of parties to achieve reasonable accuracy in such a setting).”
An overview of deep learning in medical imaging focusing on MRI (2018, background): “However, the general deep learning community are focusing a lot of attention on the issue of privacy, and new techniques and frameworks for federated learning [333]50, split learning [334, 335] and differential privacy [336, 337, 338] are rapidly improving.”
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning (2018, neutral): “Differential privacy [22], [23] has been used as a strong defense mechanism against inference attacks in the context of machine learning [24], [25], [26], [27].”
LOGAN: Membership Inference Attacks Against Generative Models (2017, background): “[46, 48] combine multiple models trained with disjoint datasets without exposing the models, while, in [47], present “defensive distillation” to reduce the effectiveness of adversarial samples on DNNs.”
Machine Learning with Membership Privacy using Adversarial Regularization (2018, methodology): “, training of models) with differential privacy guarantee [15, 16], which has recently been used in the context ofmachine learning [1, 6, 10, 40, 41].”
Measuring Forgetting of Memorized Training Examples (2022, background): “Resulting privacy concerns have led to a variety of techniques for private machine learning, including differentially private training [1, 61, 62, 23, 53], machine unlearning [5, 58, 67], and various heuristics like regularization [56], data augmentation [2] or gradient clipping [10, 72, 32].”