Data Poisoning Attacks on Federated Machine Learning

By Gan Sun, Yang Cong, Jiahua Dong, Qiang Wang, Ji Liu. 2020

293 citations recorded by the source corpus.

Federated machine learning which enables resource-constrained node devices (e.g., Internet of Things (IoT) devices and smartphones) to establish a knowledge-shared model while keeping the raw data local, could provide privacy preservation, and economic benefit by designing an effective communication protocol. However, this communication protocol can be adopted by attackers to launch data poisoning attacks for different nodes, which has been shown as a big threat to most machine learning models. Therefore, we in this article intend to study the model vulnerability of federated machine learning, and even on IoT systems. To be specific, we here attempt to attacking a popular federated multitask learning framework, which uses a general multitask learning framework to handle statistical challenges in the federated learning setting. The problem of calculating optimal poisoning attacks on federated multitask learning is formulated as a bilevel program, which is adaptive to the arbitrary selection of target nodes and source attacking nodes. We then propose a novel systems-aware optimization method, called as attack on federated learning (AT2FL), to efficiently derive the implicit gradients for poisoned data, and further attain optimal attack strategies in the federated machine learning. This is an earlier work, to our knowledge, that explores attacking federated machine learning via data poisoning. Finally, experiments on several real-world data sets demonstrate that when the attackers directly poison the target nodes or indirectly poison the related nodes via using the communication protocol, the federated multitask learning model is sensitive to both poisoning attacks.

Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety, Robustness & Interpretability

In Federated & Distributed Learning

Citations in the corpus

Poisoning Language Models During Instruction Tuning (2023, background): “For example, some work poisons traditional multi-task or federated learning models (Zhao et al., 2018; Sun et al., 2021) and others poison the pre-training or self-supervision stages and affect downstream ﬁne-tuning results (Liu et al., 2022; Carlini, 2021; Carlini & Terzis, 2022).”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, background): “…a number of strong attack methods have been explored, including scaling malicious model updates (Bagdasaryan et al., 2020), collaborative attacking (Sun et al., 2020), defense-aware attacks (Bhagoji et al., 2019; Fang et al., 2020), and adding edge-case adversarial training samples (Wang et al.,…”
Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, background): “[57] propose DPAs on federated multi-task learning where each client learns a different task.”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, background): “[41], they define a set of target nodes as those nodes (clients or server) to be compromised by the attack.”
Federated Learning: Opportunities and Challenges (2021, background): “The adversary can poison the data either by directly injecting poisoned data to the targeted device or injecting poisoned data through other devices [30].”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “Besides, SMC-based protocols can enable multiple participants to collaboratively compute an agreed-upon function without leaking input information from any participant except for what can be inferred from the outcomes of the computation [99], [100].”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “Another work instead adopts a bilevel optimization approach to poisoning multi-task federated learning [31].”
Addressing Class Imbalance in Federated Learning (2020, background): “Despite of various types of inference attacks (Melis et al. 2018; Zhu, Liu, and Han 2019; Wang et al. 2019; Sun et al. 2020), inferring information of particular clients is still extremely difficult.”