Learning privately from multiparty data

By Jihun Hamm, Yingjun Cao, Mikhail Belkin. 2016

169 citations recorded by the source corpus.

Learning a classifier from private data collected by multiple parties is an important problem that has many potential applications. How can we build an accurate and differentially private global classifier by combining locally-trained classifiers from different parties, without access to any party's private data? We propose to transfer the `knowledge' of the local classifier ensemble by first creating labeled data from auxiliary unlabeled data, and then train a global $\epsilon$-differentially private classifier. We show that majority voting is too sensitive and therefore propose a new risk weighted by class probabilities estimated from the ensemble. Relative to a non-private solution, our private solution has a generalization error bounded by $O(\epsilon^{-2}M^{-2})$ where $M$ is the number of parties. This allows strong privacy without performance loss when $M$ is large, such as in crowdsensing applications. We demonstrate the performance of our method with realistic tasks of activity recognition, network intrusion detection, and malicious URL detection.

Subfields: Artificial Intelligence, Computer Science Applications, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Towards the Science of Security and Privacy in Machine Learning (2016, neutral): “This strategy was explored in [98], [99].”
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, methodology): “…that provides differential privacy for training data in a “black-box” manner, i.e., independent of the learning algorithm, as demonstrated by Section 4 and Appendix C. • We improve upon the strategy outlined in Hamm et al. (2016) for learning machine models that protect training data privacy.”
Privacy in Deep Learning: A Survey (2020, methodology): “DPSGD [22] # # MNIST, CIFAR-10 Image Classification w/ DNN DP LSTM [77] # # Reddit Posts Language Model w/ LSTMs DP LOR [78] # # Artificial Data Logistic Regression DP ERM [79] # # Adult, KDD-99 Classification w/ ERM DP GAN [80] # # MNIST, MIMIC-III Data Generation w/ GAN DP GM [81] # # MNIST, CDR, ”
Learning Privately over Distributed Features: An ADMM Sharing Approach (2019, background): “Similar to recent literature on distributed machine learning [10, 23], ADMM [4, 3], and privacypreserving machine learning [15, 24], we assume the loss has a form ∑N i=1 li(f(D;x), Yi) = ∑N i=1 li(Dx, Yi) = l (∑M m=1D mxm ) , where we have abused the notation of l and in the second equality absorbed”
The Power of the Hybrid Model for Mean Estimation (2018, background): “In this model, some works assume that DP is achieved in the TCM [35, 50], while others assume that DP is achieved in the LM [55, 58].”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “[23] use knowledge transfer to combine a collection of models trained on individual devices into a single model, with differential privacy guarantees.”
Personalized and Private Peer-to-Peer Machine Learning (2017, methodology): “…DP algorithms have been introduced to learn a private global model, either by aggregating models trained locally by each party (Pathak et al., 2010; Hamm et al., 2016) or by perturbing the gradients and/or the objective in a distributed gradient descent algorithm (Song et al., 2013; Rajkumar and…”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, background): “Additionally, it allows parties to outsource potentially heavy computation and guarantees that they do not see model parameters during training that have to be shared, for example, in distributed settings [16, 27, 35, 36, 39].”
Scalable Private Learning with PATE (2018, background): “The first learning algorithms adapted to provide differential privacy with respect to their training data were often linear and convex (Pathak et al., 2010; Chaudhuri et al., 2011; Song et al., 2013; Bassily et al., 2014; Hamm et al., 2016).”
Federated Model Distillation with Noise-Free Differential Privacy (2020, methodology): “Knowledge of ensemble of teacher models has been used to train a student model in previous works [Hamm et al., 2016; Papernot et al., 2017; Wang et al., 2019a; Sun et al., 2020].”
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning (2018, background): “[9] leveraged knowledge transfer from ensemble local models to a global differentially private model.”
SoK: Security and Privacy in Machine Learning (2018, background): “Under different assumptions, namely the availability of public and unlabeled data whose privacy does not need to be protected, it is possible to further improve the strength of the privacy protection on sensitive (labeled) data [95], [96].”
A Study of Face Obfuscation in ImageNet (2021, background): “, 2020) or secure multi-party computation (Shokri & Shmatikov, 2015; Melis et al., 2019; Hamm et al., 2016; Pathak et al., 2010; Hamm et al., 2016).”
Private Query Release Assisted by Public Data (2020, background): “Another line of work considers the setting in which the learning task can be reduced to privately answering classification queries (Hamm et al., 2016; Papernot et al., 2017; 2018; Bassily et al., 2018; Nandi & Bassily, 2019), where the goal is to construct a differentially private classification alg”
Limits of Private Learning with Access to Public Data (2019, background): “Another line of work consider the problem of private knowledge transfer [HCB16], [PAE + 17], [PSM + 18], and [BTT18].”
Evaluating Differentially Private Machine Learning in Practice (2019, unknown): “[25] Output KDDCup99 493,000 123 1.”