Can You Really Backdoor Federated Learning?

By Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan. 2019

742 citations recorded by the source corpus.

The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of backdoor attacks and defenses for the EMNIST dataset, a real-life, user-partitioned, and non-iid dataset. We observe that in the absence of defenses, the performance of the attack largely depends on the fraction of adversaries present and the "complexity'' of the targeted task. Moreover, we show that norm clipping and "weak'' differential privacy mitigate the attacks without hurting the overall performance. We have implemented the attacks and defenses in TensorFlow Federated (TFF), a TensorFlow framework for federated learning. In open-sourcing our code, our goal is to encourage researchers to contribute new attacks and defenses and evaluate them on standard federated datasets.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Federated & Distributed Learning

Citations in the corpus

FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “To accelerate generating benchmark results on new types of adversarial attacks in FL, we include the latest robust aggregation methods presented in literature i.e. (i) norm difference clipping [23]; weak differential private (DP) [23]; (ii) RFA (geometric median) [118]; (iii) K RUM and (iv) M ULTI -”
Robust Aggregation for Federated Learning (2019, methodology): “We now compare RFA with: (a) coordinate-wise median [88] and `2 norm clipping [82] which are agnostic to the actual corruption level ρ like RFA, and, (b) trimmed mean [88] and multi-Krum [16], that require exact knowledge of the level of corruption ρ in the problem.”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “ified to bypass norm-based and statistics-based defenses for federated learning by constraining the norm and variance of gradient updates for local models [114], [115].”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, methodology): “In our experiments (Section 4), we compare Ditto with several strong defenses (median, gradient clipping (Sun et al., 2019), Krum, Multi-Krum (Blanchard et al.”
FedScale: Benchmarking Model and System Performance of Federated Learning at Scale (2021, methodology): “As for benchmarking the FL security, we follow the example setting of recent backdoor attacks [49, 50] on the OpenImage, where corrupted clients flip their ground-truth labels to poison the training.”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “Backdoor attack is one type of targeted model poisoning attacks that aims to inject a backdoor trigger into the global model without influence its performance on the main task [26], [27], [28], [29].”
Federated Learning: Opportunities and Challenges (2021, background): “Intensity of such attacks depends on the proportion of the compromised devices present and model capacity of federated learning [31] .”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, methodology): “[38], they introduce an alternative to Central DP aggregation operator, known as Weak DP, which shares the same aggregation procedure, but it does not guarantee (e, δ)-DP nor any known privacy preserving property.”
Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, methodology): “Vector-wise scaling Norm-bound [58] 86.”
Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning (2021, background): “All the robust AGRs for FL tend to remove/attenuate malicious gradients based on one or more of the following criteria: 1) distances from the benign gradients [8], [5], [31], [2], [39], 2) distributional differences with the benign gradients [5], [35], 3) difference in Lp-norms of the benign and mal”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “centralized privacy guarantees FL-LSTM [130] Local DP FL [20] LM, NN Secure Aggregation FL [23] NN CM Hybrid FL [181] LM, DT, NN CM, DP Backdoor FL [16, 174, 188] NN robustness and attacks Adversarial Lens [19] Distributed Backdoor [203] Image Reconstruction [58] RSA [100] LM Model Poison [53] LM, N”