Mitigating Sybils in Federated Learning Poisoning

By Clement Fung, Chris J. M. Yoon, Ivan Beschastnikh. 2018

670 citations recorded by the source corpus.

Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susceptible to a variety of attacks, including model poisoning, which is made substantially worse in the presence of sybils. In this paper we first evaluate the vulnerability of federated learning to sybil-based poisoning attacks. We then describe \emph{FoolsGold}, a novel defense to this problem that identifies poisoning sybils based on the diversity of client updates in the distributed learning process. Unlike prior work, our system does not bound the expected number of attackers, requires no auxiliary information outside of the learning process, and makes fewer assumptions about clients and their data. In our evaluation we show that FoolsGold exceeds the capabilities of existing state of the art approaches to countering sybil-based label-flipping and backdoor poisoning attacks. Our results hold for different distributions of client data, varying poisoning targets, and various sybil strategies. Code can be found at: this https URL

Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

How To Backdoor Federated Learning (2018, background): “models [18, 59, 60], or require participants to submit their data for centralized training [27].”
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, methodology): “[23] proposed to compute weight for each worker device according to historical local models and take the weighted average of the local models to update the global model.”
FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “…existing research focuses on developing new adversarial attack and defense techniques to make federated learning robust [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 22], and proposing methods such as differential privacy (DP) and secure multiparty computation (SMPC) to protect privacy [25,…”
Backdoor Attacks on Federated Meta-Learning (2020, methodology): “The same assumption was made in [12, 13, 19, 16, 17, 20, 21], where outliers were detected and removed according to slightly different measures taken from the distribution of benign values (benign users were assumed to be the majority).”
Practical Distributed Learning: Secure Machine Learning with Communication-Efficient Local Updates (2019, background): “On the other hand, recent work has considered attacks targeting federated learning (Bagdasaryan et al., 2018; Fung et al., 2018; Bhagoji et al., 2018), but does not propose defense techniques with provable convergence.”
A Little Is Enough: Circumventing Defenses For Distributed Learning (2019, background): “(Fung et al., 2018) proposed a de-A fense against backdoors in federated learning, but like the attack above it heavily relies on the non-i.i.d property of the data, which does not hold for distributed training.”
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System (2019, methodology): “We intend to investigate the effects of attacks and threats by incorporating recently proposed methods [31, 32, 33] for secure federated learning.”
Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory (2019, methodology): “[19] introduced the Sybil-based poisoning attacks and then identified these attacks according to client contribution similarity.”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “Data poisoning [13] means that the adversaries compromise the integrity of the training data through methods such as label flipping [55] and backdoor”
Preserving Data Privacy via Federated Learning: Challenges and Solutions (2020, background): “Alongside, to resist Sybil attacks, Fung et al. [14] proposed a new defense approach to federated learning and named it FoolsGold .”
Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices (2019, neutral): “Besides, the federated approach for training the model is susceptible to model poisoning attacks [5].”
No Peek: A Survey of private distributed deep learning (2018, background): “Data poisoning attacks on federated learning [37] where malicious users can inject false training data to negatively eﬀect the classiﬁcation performance of the model have been proposed.”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, background): “Label-flipping, which is a special case of dirty-label attacks, has been proved in [115] to be one of the”
From distributed machine learning to federated learning: a survey (2021, methodology): “Data poisoning can be realized by modifying the features [41] or the labels [133] of the input data.”
Robust Learning from Untrusted Sources (2019, background): “Fault tolerance and prevention of sybil attacks in federated learning have been considered by (Smith et al., 2017) and (Fung et al., 2018) respectively.”
SLSGD: Secure and Efficient Distributed On-device Machine Learning (2019, background): “On the other hand, recent work has considered attacks not only targeting traditional distributed SGD [28], but also federated learning [3,9,4], but do not propose defense techniques with provable convergence.”