Poisoning and Backdooring Contrastive Learning

By Nicholas Carlini, Andreas F. Terzis. 2021

225 citations recorded by the source corpus.

Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

The Curse of Recursion: Training on Generated Data Makes Models Forget (2023, background): “Recent studies have demonstrated that web-scale datasets can be poisoned by introducing malicious data into a small percentage of samples [Carlini and Terzis, 2021, Carlini et al., 2023].”
Poisoning Language Models During Instruction Tuning (2023, background): “For example, some work poisons traditional multi-task or federated learning models (Zhao et al., 2018; Sun et al., 2021) and others poison the pre-training or self-supervision stages and affect downstream ﬁne-tuning results (Liu et al., 2022; Carlini, 2021; Carlini & Terzis, 2022).”
Unsolved Problems in ML Safety (2021, background): “If an adversary uploads a few carefully crafted poisoned images [29], code snippets [156], or sentences [194] to platforms such as Flickr, GitHub or Twitter, they can inject a backdoor into future models trained on that data [10].”
Backdoor Learning: A Survey (2020, background): “D point cloud [76]–[78], semi-/self-supervised learning [79]–[81], reinforcement learning [82]–[84], model quantization [85]–[87], acoustics signal processing [88], [89], malware detection [90], [91], and other tasks [92]–[94].”
PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning (2022, background): “To the best of our knowledge, Carlini and Terzis [12] is the only work on data poisoning attacks to contrastive learning.”
Poisoning Web-Scale Training Datasets is Practical (2023, background): “5% of poisoning success is orders-of-magnitude higher than what is required in prior poisoning attacks [15].”