Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning

By Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, Daniel Ramage. IEEE Symposium on Security and Privacy, 2021

321 citations recorded by the source corpus.

While recent works have indicated that federated learning (FL) may be vulnerable to poisoning attacks by compromised clients, their real impact on production FL systems is not fully understood. In this work, we aim to develop a comprehensive systemization for poisoning attacks on FL by enumerating all possible threat models, variations of poisoning, and adversary capabilities. We specifically put our focus on un-targeted poisoning attacks, as we argue that they are significantly relevant to production FL deployments. We present a critical analysis of untargeted poisoning attacks under practical, production FL environments by carefully characterizing the set of realistic threat models and adversarial capabilities. Our findings are rather surprising: contrary to the established belief, we show that FL is highly robust in practice even when using simple, low-cost defenses. We go even further and propose novel, state-of-the-art data and model poisoning attacks, and show via an extensive set of experiments across three benchmark datasets how (in)effective poisoning attacks are in the presence of simple defense mechanisms. We aim to correct previous misconceptions and offer concrete guidelines to conduct more accurate (and more realistic) research on this topic.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/fcf391eeded299fc8761f114302b558a7a8e4df7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lichao Sun
2. Lingjuan Lyu
3. Philip S. Yu
4. Qiang Yang
5. Han Yu
6. Jun Zhao
7. Xingjun Ma
8. Charu C. Aggarwal

Citations in the corpus

• Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, methodology): “Throughout this survey, we will interchangeably use participants/clients/users to represent the participants in FL.”