Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging

By Luis Muñoz-González, Kenneth T. Co, Emil C. Lupu. 2019

229 citations recorded by the source corpus.

Federated learning enables training collaborative machine learning models at scale with many participants whilst preserving the privacy of their datasets. Standard federated learning techniques are vulnerable to Byzantine failures, biased local datasets, and poisoning attacks. In this paper we introduce Adaptive Federated Averaging, a novel algorithm for robust federated learning that is designed to detect failures, attacks, and bad updates provided by participants in a collaborative model. We propose a Hidden Markov Model to model and learn the quality of model updates provided by each participant during training. In contrast to existing robust federated learning schemes, we propose a robust aggregation rule that detects and discards bad or malicious local model updates at each training iteration. This includes a mechanism that blocks unwanted participants, which also increases the computational and communication efficiency. Our experimental evaluation on 4 real datasets show that our algorithm is significantly more robust to faulty, noisy and malicious participants, whilst being computationally more efficient than other state-of-the-art robust federated learning methods such as Multi-KRUM and coordinate-wise median.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Martin Jaggi
2. Sai Praneeth Karimireddy
3. Lie He
4. Yang Liu
5. Tianjian Chen
6. Azzam Mourad
7. Yuxi Fan
8. Li Li

Citations in the corpus

• Backdoor attacks and defenses in feature-partitioned collaborative learning (2020, background): “Unlike Byzantine attacks (Muñoz-González et al., 2019), its goal is not to prevent convergence.”
• Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning (2021, background): “6) Adaptive federated average (AFA): AFA [33] removes malicious gradients based on their cosine-similarities with a benign gradient.”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “the authors of [147] propose a Byzantine-robust federated learning approach called Adaptive Federated Averaging.”
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, unknown): “• Adaptive Federated Averaging (AFA) [92].”
• A Survey on federated learning* (2020, methodology): “Muñoz-González&Lupu discussed adaptive FL to grub abnormal updates via a Hidden Markov Model to evaluate model quality [64].”
• FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping (2020, unknown): “, [9], [12], [29], [34], [44], [47], [48], [31]) that aim to tolerate Byzantine client failures.”
• Secure Byzantine-Robust Machine Learning (2020, methodology): “[28] uses HMM to detect and exclude Byzantine workers for federated learning.”