MetaPoison: Practical General-purpose Clean-label Data Poisoning

By W. Ronny Huang, Jonas Geiping, Liam H. Fowl, Gavin Taylor, Tom Goldstein. 2020

237 citations recorded by the source corpus.

Data poisoning--the process by which an attacker takes control of a model by making imperceptible changes to a subset of the training data--is an emerging threat in the context of neural networks. Existing attacks for data poisoning have relied on hand-crafted heuristics. Instead, we pose crafting poisons more generally as a bi-level optimization problem, where the inner level corresponds to training a network on a poisoned dataset and the outer level corresponds to updating those poisons to achieve a desired behavior on the trained model. We then propose MetaPoison, a first-order method to solve this optimization quickly. MetaPoison is effective: it outperforms previous clean-label poisoning methods by a large margin under the same setting. MetaPoison is robust: its poisons transfer to a variety of victims with unknown hyperparameters and architectures. MetaPoison is also general-purpose, working not only in fine-tuning scenarios, but also for end-to-end training from scratch with remarkable success, e.g. causing a target image to be misclassified 90% of the time via manipulating just 1% of the dataset. Additionally, MetaPoison can achieve arbitrary adversary goals not previously possible--like using poisons of one class to make a target image don the label of another arbitrarily chosen class. Finally, MetaPoison works in the real-world. We demonstrate successful data poisoning of models trained on Google Cloud AutoML Vision. Code and premade poisons are provided at this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Privacy & Security

Citations in the corpus

Poisoning Language Models During Instruction Tuning (2023, methodology): “Thus, as is common in existing attacks (e.g., Tramèr et al. 2018; Huang et al. 2020; Wallace et al. 2020), we optimize the poison examples using a proxy model that is related to the target model.”
Concealed Data Poisoning Attacks on NLP Models (2021, methodology): “Instead, we follow past work on poisoning vision models (Huang et al., 2020), which builds upon similar ideas in other areas such as meta learning (Finn et al., 2017) and distillation (Wang et al., 2018), and approximate the inner training loop using a small number of gradient descent steps.”
Poisoning Web-Scale Training Datasets is Practical (2023, unknown): “…there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, methodology): “Witches’ Brew [47] improves on MetaPoison by introducing a “gradient alignment” objective that encourages the gradient of the loss on poison data to match the gradient of the adversary’s target loss.”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, unknown): “…(including data poisoning and model poisoning) have been extensively studied in prior work (Biggio et al., 2012; Gu et al., 2017; Chen et al., 2017; Shafahi et al., 2018; Liu et al., 2018; Huang et al., 2020; Xie et al., 2020; Wang et al., 2020; Dumford & Scheirer, 2018; Huang et al., 2020).”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, methodology): “[50] proposed the MetaPoison, which is a first-order optimization method that approximates the bilevel problem via meta-learning and constructs poisoning samples against deep learning models.”
Customizing Triggers with Concealed Data Poisoning (2020, unknown): “Instead, we follow past work on poisoning vision models (Huang et al., 2020), which builds upon similar ideas in other areas such as meta learning (Finn et al., 2017) and distillation (Wang et al., 2018), and approximate the inner training loop using a small number of gradient descent steps.”
Unveiling Security, Privacy, and Ethical Concerns of ChatGPT (2023, background): “Training data poisoning attacks [22] pose a signiﬁcant threat to the ﬁeld of AI as they involve contaminating the training data used to train machine learning models, leading to erroneous outputs and unreliable decision-making.”