Multiparty Differential Privacy via Aggregation of Locally Trained Classifiers

By Manas A. Pathak, Shantanu D. Rane, Bhiksha Raj. 2010

206 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Privacy & Security

Citations in the corpus

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “Several works have adopted DP as an efficient defense mechanism such as [5, 7, 11, 12, 18, 23, 24, 37, 53, 60, 65, 72, 86].”
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, methodology): “Intuitively, this strategy ensures that the student does not depend on the details of any single sensitive training data point (e.g., of any single user), and, thereby, the privacy of the training data is protected even if attackers can observe the student’s internal model parameters.”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “[47] present a differentially private global classifier hosted by a trusted third-party and based on locally trained classifiers held by separate, mutually distrusting parties.”
Personalized and Private Peer-to-Peer Machine Learning (2017, methodology): “Existing approaches for privacy-preserving distributed learning (see e.g., Pathak et al., 2010; Rajkumar and Agarwal, 2012; Shokri and Shmatikov, 2015; Huang et al., 2015) rely on a central (sometimes trusted) server, assume the local data distribution is the same for all users and/or are designed…”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, background): “Additionally, it allows parties to outsource potentially heavy computation and guarantees that they do not see model parameters during training that have to be shared, for example, in distributed settings [16, 27, 35, 36, 39].”
Scalable Private Learning with PATE (2018, background): “The first learning algorithms adapted to provide differential privacy with respect to their training data were often linear and convex (Pathak et al., 2010; Chaudhuri et al., 2011; Song et al., 2013; Bassily et al., 2014; Hamm et al., 2016).”
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning (2018, methodology): “[10] proposed a DP-based global classifier by aggregating the locally trained classifiers.”
Learning privately from multiparty data (2016, methodology): “…batch: classifier trained using all data ignoring privacy • soft: private ensemble using soft-labels (Algorithm 2) • avg: parameter averaging (Pathak et al., 2010) • vote: private ensemble using majority voting (Algo- rithm 1) • indiv: individually trained classifier using local…”
Overcoming Catastrophic Forgetting by Incremental Moment Matching (2017, background): “The idea of mean-IMM is commonly used in shallow networks [19, 20].”
A Study of Face Obfuscation in ImageNet (2021, background): “…to train a model collectively without sharing their private data using federated learning (McMahan et al., 2017; Bonawitz et al., 2017; Li et al., 2020) or secure multi-party computation (Shokri & Shmatikov, 2015; Melis et al., 2019; Hamm et al., 2016; Pathak et al., 2010; Hamm et al., 2016).”
Auror: defending against poisoning attacks in collaborative deep learning systems (2016, background): “Indirect collaborative learning addresses both these concerns [35, 39, 43].”
On the Protection of Private Information in Machine Learning Systems: Two Recent Approches (2017, background): “They belong in a line of work on knowledge aggregation and transfer for privacy [27], [28], [17].”
Differentially private distributed logistic regression using private and public data (2014, background): “For example, Pathak, Rane, Raj et al. [19] suggested running local models on each private data set and aggregating estimated parameters.”
Generative Knowledge Transfer for Neural Language Models (2016, background): “Several studies trained private classifiers to produce a final distributable classifier in various ways, such as averaging the model parameters of teacher networks (Pathak et al., 2010), training hard labels by voting the ensemble of all the teachers (Papernot et al.”
Differentially Private Hypothesis Transfer Learning (2018, background): “[31] addressed the hypothesis ensemble problem by simple parameter averaging and publishing the aggregated hypothesis in a differentially private manner.”
Privacy-preserving deep learning (2015, background): “Aggregation of independently trained neural networks using differential privacy and secure multi-party computation is suggested in [37].”
When Machine Learning Meets Privacy (2020, methodology): “DPSGD Distributed [2] ✓ Obfuscation DPSGD Centralised [100] ✓ Obfuscation multi-round DP Centralised [95] ✓ Obfuscation DP-FedAvg Distributed [179] ✓ Obfuscation Training data obfuscation Centralised [121] ✓ Aggregation/Obfuscation DP+Aggregation Distributed [69,94] ✓ Aggregation Federated learning ”
FDML: A Collaborative Machine Learning Framework for Distributed Features (2019, background): “Relatively earlier, differential privacy has also been applied to collaborative machine learning [23, 24], with an inherent tradeoff between privacy and utility of the trained model.”