Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection

By Andrea Paudice, Luis Muñoz-González, András György, Emil C. Lupu. 2018

166 citations recorded by the source corpus.

Machine learning has become an important component for many systems and applications including computer vision, spam filtering, malware and network intrusion detection, among others. Despite the capabilities of machine learning algorithms to extract valuable information from data and produce accurate predictions, it has been shown that these algorithms are vulnerable to attacks. Data poisoning is one of the most relevant security threats against machine learning systems, where attackers can subvert the learning process by injecting malicious samples in the training data. Recent work in adversarial machine learning has shown that the so-called optimal attack strategies can successfully poison linear classifiers, degrading the performance of the system dramatically after compromising a small fraction of the training dataset. In this paper we propose a defence mechanism to mitigate the effect of these optimal poisoning attacks based on outlier detection. We show empirically that the adversarial examples generated by these attack strategies are quite different from genuine points, as no detectability constrains are considered to craft the attack. Hence, they can be detected with an appropriate pre-filtering of the training dataset.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Backdoor Attacks on Federated Meta-Learning (2020, background): “Given the requirement that users of federated learning share only updates of the model instead of training data [1], techniques certifying that training examples are correctly classified [34, 35] are not applicable in this context, and thus defense remains a challenging problem.”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “A common and often effective defense against data poisoning attacks is data sanitization, which use anomaly detectors to filter out training points that look suspicious (Hodge & Austin, 2004; Cretu et al., 2008; Paudice et al., 2018).”
Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach (2019, methodology): “For data sanitization, we add adversarial examples to the datasets by using Lasso classifier poisoning [7] and then defend the attack with clustering-based anomaly detection using the k-means algorithm.”
Deep Learning for Anomaly Detection (2020, background): “Last, other interesting applications include detection of adversarial examples [51, 119], anti-spoofing in biometric systems [46, 121], and early detection of rare catastrophic events (e.”
Data Poisoning Attacks Against Federated Learning Systems (2020, background): “Similarly, server-side defenses that rely on ﬁltering and eliminating poison instances through anomaly detection or k-NN [36,37] are inapplicable to FL since the server only observes parameter updates from FL participants, not their individual instances.”
A Full Dive Into Realizing the Edge-Enabled Metaverse: Visions, Enabling Technologies, and Challenges (2022, background): “The authors in [232] propose a defense mechanism that allows the edge devices to pre-filter the training dataset to remove the adversarial examples.”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “In a similar spirit, [132] proposes a data pre-filtering approach with outlier detection for linear classifiers.”
Security and Privacy Issues in Deep Learning (2018, methodology): “The recent approaches include outlier detections to eliminate [13] or re-label [14] the suspected poisoned examples.”
PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning (2022, background): “Other pre-processing defenses aim to detect poisoning training examples based on anomaly detection [13, 17, 41], which can be broken by strong, adaptive attacks [34].”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, methodology): “Literature Model-protected Strategy Adversarial Setting Year Techniques Goals [4] cross-models data-driven cross-attacks UnT 2010 [33] logistic regression data-driven DM UnT 2014 [72] cross-models data-driven LM UnT 2015 [99] cross-models data-driven LM UnT 2018 [98] cross-models data-driven cross-a”
Fawkes: Protecting Privacy against Unauthorized Deep Learning Models (2020, background): “Since cloaking is a form of data poisoning, prior work on detecting poisoning attacks [10, 18, 39, 45, 48, 57] could be helpful.”