Contamination Attacks and Mitigation in Multi-Party Machine Learning

By Jamie Hayes, O. Ohrimenko. 2018

76 citations recorded by the source corpus.

Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time. Distinct parties may want to combine their local data to gain the benefits of a model trained on a large corpus of data. We consider such a case: parties get access to the model trained on their joint data but do not see each others individual datasets. We show that one needs to be careful when using this multi-party model since a potentially malicious party can taint the model by providing contaminated data. We then show how adversarial training can defend against such attacks by preventing the model from learning trends specific to individual parties data, thereby also guaranteeing party-level membership privacy.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Privacy & Security

Citations in the corpus

How To Backdoor Federated Learning (2018, background): “models [18, 59, 60], or require participants to submit their data for centralized training [27].”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “ks [58], and inferring hyperparameters with hyperparameter stealing attacks [60]. In response to these attacks, the privacy community has designed defenses to prevent privacy leakage of training data [1, 19, 36, 46] or the target model [26, 31]. However, one important limitation of current machine l”
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer (2019, background): “The earlier attacks aim to jeopardize the overall accuracy of the ﬁnal model/s [11], [23], [50], while the latter attacks aim to mis-classify only a speciﬁc set of samples of the attacker’s choice at the test time [10], [12].”
Distributed Differentially Private Averaging with Improved Utility and Robustness to Malicious Parties (2020, background): “Second, malicious users may send incorrect results to bias the learned model in arbitrary ways [45, 10, 3].”
Poisoning Attack in Federated Learning using Generative Adversarial Nets (2019, background): “We notice that it is possible to construct the poisoning attack in federated learning for the following reasons [6]: 1) there are plenty of participants in federated learning system, it is most likely contains one or more malicious users; 2) since the participants’ local data and training process ar”
On the Accuracy of Influence Functions for Measuring Group Effects (2019, background): “, 2018); or, in a multi-party learning setting, ensure that no individual user has too much influence on the joint model (Hayes and Ohrimenko, 2018).”
Membership Inference Attacks Against Adversarially Robust Deep Learning Models (2019, background): “Along with finding novel attacks again deep learning models, the research community has also proposed defense approaches to resolve both security issues [17]–[21] and privacy issues [22]–[25].”
Understanding Distributed Poisoning Attack in Federated Learning (2019, background): “[4] evaluate contamination attacks in multi-party machine learning.”