Support vector machines under adversarial label contamination

By Huang Xiao, Battista Biggio, Blaine Nelson, Han Xiao, Claudia Eckert, Fabio Roli. 2015

241 citations recorded by the source corpus.

Machine learning algorithms are increasingly being applied in security-related tasks such as spam and malware detection, although their security properties against deliberate attacks have not yet been widely understood. Intelligent and adaptive attackers may indeed exploit specific vulnerabilities exposed by machine learning techniques to violate system security. Being robust to adversarial data manipulation is thus an important, additional requirement for machine learning algorithms to successfully operate in adversarial settings. In this work, we evaluate the security of Support Vector Machines (SVMs) to well-crafted, adversarial label noise attacks. In particular, we consider an attacker that aims to maximize the SVM?s classification error by flipping a number of labels in the training data. We formalize a corresponding optimal attack strategy, and solve it by means of heuristic approaches to keep the computational complexity tractable. We report an extensive experimental analysis on the effectiveness of the considered attacks against linear and non-linear SVMs, both on synthetic and real-world datasets. We finally argue that our approach can also provide useful insights for developing more secure SVM learning algorithms, and also novel techniques in a number of related research areas, such as semi-supervised and active learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Can You Really Backdoor Federated Learning? (2019, background): “attacks (5; 28; 33; 25; 18), the attacker can change a subset of all the training samples which is unknown to the learner.”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “In a typical label-flip attack, the attacker gets to change some fraction of the labels y of the training set but is unable to change the features x (Biggio et al., 2011; Xiao et al., 2012, 2015).”
Spectral Signatures in Backdoor Attacks (2018, background): “One well-studied setting for such training set attacks is data poisoning [3, 34, 25, 18, 31].”
Certified Defenses for Data Poisoning Attacks (2017, methodology): “Our contribution relates to the long line of work on data poisoning attacks; beyond linear classiﬁers, others have studied the LASSO (Xiao et al., 2015a), clustering (Biggio et al., 2013; 2014c), PCA (Rubinstein et al., 2009), topic modeling (Mei and Zhu, 2015a), collaborative ﬁltering (Li et al.,…”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, background): “In this way our work is similar to work on targeted poison attacks [2, 4, 21, 23, 42, 43] in machine learning, where the aim is to degrade the performance of a model.”
Model-Reuse Attacks on Deep Learning Systems (2018, background): “(i) Poisoning attacks, in which the adversary pollutes the training data to eventually compromise the ML systems [9, 41, 61, 62].”
Data Poisoning Attacks against Autoregressive Models (2016, background): “Support vector machines, in particular, have received a great deal of attention (Biggio and Laskov 2012; Biggio et al. 2014; Xiao et al. 2014; Xiao, Xiao, and Eckert 2012).”
A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View (2018, background): “extended the adversary model to attack against some black-box linear learning models such as SVM and logistic regression (LR) [51].”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, methodology): “Literature Adversarial Setting Target Model Poisoning Technique Application Evaluation Year Knowledge Capability [106] N-K to P-K IN PCA data inserting intrusion detection ROC 2009 [10] N-K to L-K MO SVM label manipulation binary classification accuracy 2011 [11] P-K IN SVM data manipulation image c”
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks (2020, background): “The objective for data poisoning attack is to disrupt the proper training of DNN models by reducing their test-time performance on all or a specific subset of test examples [5,7,31,46,47,69].”
Hyperparameter Selection under Localized Label Noise via Corrupt Validation (2017, background): “1 Some recent work has considered cases of feature-dependent label noise [8, 11–13] or even adversarial label noise [14, 15] but is either primarily theoretical [8, 12, 14] or model-specific [8, 11, 13–15], where our proposed label noise model is empirically-driven and model-agnostic as will be desc”
Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, methodology): “We build on the classic label flipping data poisoning attack [45], [64], [65] designed for centralized ML.”
EagleEye: Attack-Agnostic Defense against Adversarial Inputs (Technical Report) (2018, background): “cious attacks [3, 4, 20]. Two primary threat models are considered in literature: (i) poisoning attacks, in which the attackers pollute the training data to eventually compromise the learning systems [7, 35, 45], and (ii) evasion attacks, in which the attackers modify the input data at test time to ”
Hidden Trigger Backdoor Attacks (2019, methodology): “More advanced methods were proposed in (Xiao et al. 2015; Koh and Liang 2017; Mei and Zhu 2015; Burkard and Lagesse 2017; Newell et al. 2014) which change the result of the classiﬁer on the clean data as well.”
Evaluating Differentially Private Machine Learning in Practice (2019, background): “In addition to these attacks, several attacks have been proposed that require an adversary that can actively interfere with the model training process [5, 51, 72, 74].”
Backdoor attacks against learning systems (2017, background): “(i) Poisoning attacks, in which the adversary pollutes the training data to eventually compromise the sys-tem [39], [40].”
Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder (2020, background): “A predecessor of BP, called data poisoning, also poisons the training dataset of the victim model (Nelson et al., 2008; Biggio et al., 2012; Xiao et al., 2015; Mei and Zhu, 2015; Koh and Liang, 2017; Steinhardt et al., 2017) with the aim of reducing the model’s generalization.”