Generative Poisoning Attack Method Against Neural Networks

By Chaofei Yang, Qing Wu, Hai Helen Li, Yiran Chen. 2017

230 citations recorded by the source corpus.

Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) has been extensively studied before, there is still very limited knowledge about how such attack can be implemented on neural networks (NN), especially DNNs. In this work, we first examine the possibility of applying traditional gradient-based method (named as the direct gradient method) to generate poisoned data against NNs by leveraging the gradient of the target model w.r.t. the normal data. We then propose a generative method to accelerate the generation rate of the poisoned data: an auto-encoder (generator) used to generate poisoned data is updated by a reward function of the loss, and the target NN model (discriminator) receives the poisoned data to calculate the loss w.r.t. the normal data. Our experiment results show that the generative method can speed up the poisoned data generation rate by up to 239.38x compared with the direct gradient method, with slightly lower model accuracy degradation. A countermeasure is also designed to detect such poisoning attack methods by checking the loss of the target model.

Subfields: Artificial Intelligence, Computer Networks and Communications, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Safety, Robustness & Interpretability

Citations in the corpus

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering (2018, background): “Attacks : Yang et al. (2017) described how generative neural networks could be used to craft poisonous data that maximizes the error of the trained classiﬁer, while Mu˜noz-Gonz´alez et al. (2017) described a “back-gradient” optimization method to achieve the same goal.”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “sequently been extended to linear and logistic regression (Mei and Zhu, 2015b), topic modeling (Mei and Zhu, 2015a), collaborative ltering (Li et al., 2016), and neural networks (Koh and Liang, 2017; Yang et al., 2017; Munoz-Gonz~ alez et al., 2017). Labelip attacks. All of the attacks presented in ”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “After presenting the details of the threat model above, we emphasize again the differences between this work and a long line of existing work studying poisoning attacks and defenses [10], [38], [12], [11], [68], [44], [9], [35], [70], [39], [61].”
Certified Defenses for Data Poisoning Attacks (2017, methodology): “…(Xiao et al., 2015a), clustering (Biggio et al., 2013; 2014c), PCA (Rubinstein et al., 2009), topic modeling (Mei and Zhu, 2015a), collaborative ﬁltering (Li et al., 2016), neural networks (Yang et al., 2017), and other models (Mozaffari-Kermani et al., 2015; Vuurens et al., 2011; Wang, 2016).”
Bypassing Backdoor Detection Algorithms in Deep Learning (2019, background): “The attacker can degrade the model’s predictive power, or change its behavior according to the adversary’s objective, by poisoning the training set [8], [19], [11], [28], or its parameters [2].”
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks (2018, methodology): “Targeted backdoor attacks [Chen et al., 2017] with few resources ( ∼ 50 training examples) have been recently shown to cause the classiﬁer to fail for special test examples.”
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks (2018, background): “Existing attacks [51, 29, 20] often assume full control over the training process of victim classifiers and have similar shortcomings to white-box attacks.”
Neural Trojans (2017, methodology): “Yang et al. proposed a data gradient method to generate poisoned training data [16].”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, methodology): “Several methods thus propose to bypass this problem by training generative models such as GANs and autoencoders to produce poisons that mimic those generated by bilevel methods [51], [52].”
A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View (2018, methodology): “, SVM [43], [44], neural networks [45], Latent Dirichlet Allocation [46], principle component analysis (PCA) and LASSO [47].”
Security and Privacy Issues in Deep Learning (2018, methodology): “[92] introduce the possibility of applying the gradient-based method to DNNs, and they develop a generative method inspired by the concept of GAN [40].”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, background): “In addition to these attacks, several poisoning and adversarial training attacks have been proposed [4, 51, 73, 75] which require an adversary that can actively interfere with the model training process.”
Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, background): “1) Data Poisoning Attacks (DPAs): DPAs have been studied mainly for centralized ML [17], [44], [64], [65], [69], and no prior work has studied untargeted DPAs that are tailored to FL settings.”
Trojaning Attack on Neural Networks (2018, methodology): “Poisoning attacks on machine learning models also have been studied by many researchers [41, 59, 61].”
Data Poisoning Attacks Against Federated Learning Systems (2020, methodology): “Several poisoning attacks were developed for popular ML models including SVM [6,12,44,45,50,52], regression [19], dimensionality reduction [51], linear classiﬁers [12,23,57], unsupervised learning [7], and more recently, neural networks [12,30,42,45,53,58].”
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021, background): “Data poisoning attacks (Biggio et al., 2012; Yang et al., 2017; Steinhardt et al., 2017) share some similarities with backdoor attacks based on training data poisoning.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, methodology): “A similar technique was also used in poisoning attacks against centralized machine learning [142].”
A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning (2019, background): “Early poisoning attacks to the training set aimed at generic misclassiﬁcation [3], [4]; more recently, attacks have been proposed that focus on target misclassiﬁcation [5], [6].”