Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
By Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein. 2020
406 citations recorded by the source corpus.
As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Luke S. Zettlemoyer
- Quoc V. Le
- Mohit Bansal
- Pieter Abbeel
- Percy Liang
- Jianfeng Gao
- Yejin Choi
In Safety, Robustness & Interpretability
- Jan Leike
- Yong Jiang
- Baoyuan Wu
- Goutham Ramakrishnan
- Ilya Sutskever
- Aws Albarghouthi
- Zhifeng Li
- Dario Amodei
In Signal Processing
- Heewoo Jun
- Ilya Sutskever
- Wojciech Zaremba
- Mark Chen
- Pamela Mishkin
- Yong Jiang
- Baoyuan Wu
- Harrison Edwards
Citations in the corpus
- Evaluating Large Language Models Trained on Code (2021, background): “Similarly, the public data should generally be treated as untrusted, as previous work (Goldblum et al., 2021; Schuster et al., 2020) has found that attackers may be able to corrupt training data to trigger specific model behaviors at runtime.”
- Backdoors in Neural Models of Source Code (2020, background): “Backdoors: Backdoors are training-time attacks on deep learning models, in which an adversary manipulates the model to make malicious predictions in the presence of input triggers (See [14] and [15] for comprehensive surveys).”
- Backdoor Learning: A Survey (2020, background): “Different from concurrent papers which summarized only limited research [31], [32], [33] or classified existing methods simply by the adversary capabilities [34], [35], [36], we provide a brief yet comprehensive review as well as the taxonomy for existing methods based on their characteristics and p”