Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
By Minghong Fang, Xiaoyu Cao, Jinyuan Jia, Neil Zhenqiang Gong. USENIX Security Symposium, 2019
1,621 citations recorded by the source corpus.
In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We assume an attacker has compromised some client devices, and the attacker manipulates the local model parameters on the compromised client devices during the learning process such that the global model has a large testing error rate. We formulate our attacks as optimization problems and apply our attacks to four recent Byzantine-robust federated learning methods. Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices. We generalize two defenses for data poisoning attacks to defend against our local model poisoning attacks. Our evaluation results show that one defense can effectively defend against our attacks in some cases, but the defenses are not effective enough in other cases, highlighting the need for new defenses against our local model poisoning attacks to federated learning.
Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security
View Paper: https://arxiv.org/abs/1911.11815
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Lichao Sun
- Qiang Yang
- Philip S. Yu
- Lingjuan Lyu
- Neil Zhenqiang Gong
- Amir Salman Avestimehr
- Dusist Niyato
- Han Yu
In Federated & Distributed Learning
- Dusist Niyato
- Qiang Yang
- Amir Salman Avestimehr
- Lingjuan Lyu
- Bingsheng He
- Qinbin Li
- Virginia Smith
- Tian Li
In Privacy & Security
- Lingjuan Lyu
- Dusist Niyato
- Neil Zhenqiang Gong
- Basak Guler
- Amir Salman Avestimehr
- Jinhyun So
- Lichao Sun
- Philip S. Yu
Citations in the corpus
- Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning (2020, background): “Model poisoning attacks on federated learning architectures have been analyzed in [38], [39].”
- Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, background): “, model poisoning attacks [7], [23], [55].”
- FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clients (2022, methodology): “Untargeted Model Poisoning Attack: Untargeted model poisoning attacks aim to corrupt the global model such that it has a low accuracy for indiscriminate testing inputs.”
- Biscotti: A Blockchain System for Private and Secure Federated Learning (2021, background): “However, these techniques can be manipulated by adversaries [18], [48].”
- Robust Federated Recommendation System (2020, background): “[13] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong.”
- Privacy-Enhanced Federated Learning Against Poisoning Adversaries (2021, background): “However, Fang et al. [40] suggested that both of the above aggregation rules are not effective enough for an adversary with a certain amount of knowledge, who can carefully design a set of similar gradients to befuddle the aggregation rules.”
- From distributed machine learning to federated learning: a survey (2021, background): “With Byzantine failures, some computing resources (bad users) are manipulated by attackers during the distributed training process, which significantly degrades the performance of the global model in terms of test error [41].”
- Differential Privacy for Deep and Federated Learning: A Survey (2022, background): “In addition, using this decentralized strategy to aggregate local gradients make the FL network vulnerable to label-flipping and data poisoning attacks [124], [125], besides, it is difficult for the server to distinguish malicious from legitimate clients.”
- Data Poisoning Attacks Against Federated Learning Systems (2020, background): “While model poisoning may remain successful despite Byzantine-resilient aggregation [4,14,20], it is unclear whether optimal data poisoning attacks can be found to circumvent an individual Byzantineresilient scheme, or whether one data poisoning attack may circumvent multiple Byzantine-resilient sch”
- PPFL: privacy-preserving federated learning with trusted execution environments (2021, background): “In FL, however, there are also non-honest attacks such as backdoor [4, 67] or poisoning attacks [15], whose goal is to actively change the global model behavior, e.”
- PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning (2022, background): “As a result, other than tampering with their data, malicious clients (fake clients or compromised genuine ones) can also tamper with the computation process, which result in stronger poisoning attacks [10, 18].”
- Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, neutral): “of local parameters of benignworker devices [70].”
- FLAME: Taming Backdoors in Federated Learning (2022, methodology): “Since [20] Approximating HDBSCAN by DBSCAN.”
- Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges (2021, background): “In particular, a shared global model is learned using distributed local datasets under the control of a blockchain which also provides model verification in a decentralized manner.”