Data Poisoning Attacks Against Federated Learning Systems

By Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, Ling Liu. European Symposium on Research in Computer Security, 2020

946 citations recorded by the source corpus.

Federated learning (FL) is an emerging paradigm for distributed training of large-scale deep neural networks in which participants’ data remains on their own devices with only model updates being shared with a central server. However, the distributed nature of FL gives rise to new threats caused by potentially malicious participants. In this paper, we study targeted data poisoning attacks against FL systems in which a malicious subset of the participants aim to poison the global model by sending model updates derived from mislabeled data. We first demonstrate that such data poisoning attacks can cause substantial drops in classification accuracy and recall, even with a small percentage of malicious participants. We additionally show that the attacks can be targeted, i.e., they have a large negative impact only on classes that are under attack. We also study attack longevity in early/late round training, the impact of malicious participant availability, and the relationships between the two. Finally, we propose a defense strategy that can help identify malicious participants in FL to circumvent poisoning attacks, and demonstrate its effectiveness.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/55791f97a6ad3cf4173fa2d0900aeed3022e2bbb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Robust Privacy-Preserving Federated Learning Model Against Model Poisoning Attacks (2024, background): “These attacks have impeded the advancement of PPFL [28], [29], introducing encrypted poisonous updates rather than poisonous samples, making them more clandestine and potent [28], [29].”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “Attack methods [62, 136, 203, 241] in federated learning settings should be studied to improve the attack robustness of federated systems.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “In most scenarios, the adversary can only control a small number of the clients to carry out poisoning attacks [8, 123, 136].”
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping (2020, background): “FL is also vulnerable to data poisoning attacks [38], i.”
Federated Learning: Opportunities and Challenges (2021, background): “By targeted it means, the adversaries want to influence on the prediction of a subset of classes while deteriorating the global model accuracy[33].”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “training system comprises many users that sample from diverse distributions unknown to the attacker [32].”
Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, background): “Targeted attacks [7], [59] aim to misclassify speciﬁc sets/- classes of input, hence they are “discriminate.””
Decentralized Federated Learning: A Survey and Perspective (2023, background): “For security issues, there are different directions of research, including malicious attacks [100], data poisoning [101], anomaly detection [102], and privacy protection [81].”