Poisoning Attacks against Support Vector Machines

By Battista Biggio, Blaine Nelson, Pavel Laskov. 2012

1,848 citations recorded by the source corpus.

We investigate a family of poisoning attacks against Support Vector Machines (SVM). Such attacks inject specially crafted training data that increases the SVM's test error. Central to the motivation for these attacks is the fact that most learning algorithms assume that their training data comes from a natural or well-behaved distribution. However, this assumption does not generally hold in security-sensitive settings. As we demonstrate, an intelligent adversary can, to some extent, predict the change of the SVM's decision function due to malicious input and use this ability to construct malicious data. The proposed attack uses a gradient ascent strategy in which the gradient is computed based on properties of the SVM's optimal solution. This method can be kernelized and enables the attack to be constructed in the input space even for non-linear kernels. We experimentally demonstrate that our gradient ascent procedure reliably identifies good local maxima of the non-convex validation error surface, which significantly increases the classifier's test error.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In Safety, Robustness & Interpretability

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “A data poisoning [1,9,13,27,33,52,55, 73] attack modifies the training data.”
The Curse of Recursion: Training on Generated Data Makes Models Forget (2023, background): “Poisoning attacks are crafted and inserted during training in order to degrade the model’s performance when deployed [Biggio et al., 2012].”
Towards the Science of Security and Privacy in Machine Learning (2016, methodology): “In the settings of ofﬂine learning, Biggio et al. introduce an attack that also inserts inputs in the training set [46].”
Synthetic and Natural Noise Both Break Neural Machine Translation (2017, background): “2 ADVERSARIAL EXAMPLES The growing literature on adversarial examples has demonstrated how dangerous it can be to use brittle machine learning systems so pervasively in the real world (Biggio et al., 2012; Szegedy et al., 2014; Goodfellow et al., 2015; Mei & Zhu, 2015).”
Understanding Black-box Predictions via Influence Functions (2017, methodology): “Biggio et al. (2012) constructed a dataset poisoning attack against a linear SVM on a two-class MNIST task, but had to modify the training points in an obviously distinguishable way to be effective.”
How To Backdoor Federated Learning (2018, background): ““Traditional” poisoning attacks compromise the training data to change the model’s behavior at test time [4, 30, 42, 58, 63].”
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, background): “Attacker’s goal: Like many studies on poisoning attacks [7, 8, 30, 33, 50, 62, 65], we consider an attacker’s goal is to manipulate the learnt global model such that it has a high error rate indiscriminately for testing examples.”
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks (2015, background): “More generally, attacks against machine learning models can be partitioned by execution time: during training [43], [44] or at test time [14] when the model is used to make predictions.”
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “A common trick used to solve the given bilevel optimization problems is to replace the inner optimization by its equilibrium conditions [24, 27, 29, 31].”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “Across most FL settings, it is assumed that there is no single, central authority that owns or verifies the training data or user hardware, and it has been argued by many recent studies that FL lends itself to new adversarial attacks during decentralized model training [12, 13, 14, 15, 16, 17, 18, 1”
Privacy in Deep Learning: A Survey (2020, background): “There are few other security vulnerabilities which can be exploited in a deep learning model such as adversarial attacks [32], data poisoning [33].”
Can You Really Backdoor Federated Learning? (2019, background): “attacks (5; 28; 33; 25; 18), the attacker can change a subset of all the training samples which is unknown to the learner.”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “In contrast, poisoning attacks target the training process by maliciously modifying part of training data to cause the trained model to misbehave on some test inputs [6, 27, 43].”
Data Poisoning against Differentially-Private Learners: Attacks and Defenses (2019, background): “[Biggio et al., 2012; Mei and Zhu, 2015; Xiao et al., 2015]) because the differentially-private learner produces a stochastic model, so the objective takes the expected value of C.”
A Little Is Enough: Circumventing Defenses For Distributed Learning (2019, background): “Backdooring [4, 8, 19], is an attack in which the attacker manipulates the model at training time so that it will produce the attacker-chosen target at inference time.”
Analyzing Federated Learning through an Adversarial Lens (2018, neutral): “We focus on an adversary which directly performs model poisoning instead of data poisoning (Biggio et al., 2012; Rubinstein et al., 2009; Mei & Zhu, 2015; Xiao et al., 2015; Mei & Zhu, 2015; Koh & Liang, 2017; Chen et al., 2017a; Jagielski et al., 2018) as the agents’ data is never shared with the…”
Dataset Distillation (2018, neutral): “Compared to prior data poisoning attacks (Biggio et al., 2012; Li et al., 2016; Muñoz-González et al., 2017; Koh & Liang, 2017), our approach crucially does not require the poisoned training data to be stored and trained on repeatedly.”
Stronger data poisoning attacks break data sanitization defenses (2018, methodology): “vely modies each attack point (x;~ y~) 2D p by following the gradient of the test loss with respect to x~. This type of gradient-based data poisoning attack was rst studied in the context of SVMs by Biggio et al. (2012), and has subsequently been extended to linear and logistic regression (Mei and Z”