Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics

By Xin Li, Fuxin Li. 2016

393 citations recorded by the source corpus.

Deep learning has greatly improved visual recognition in recent years. However, recent research has shown that there exist many adversarial examples that can negatively impact the performance of such an architecture. This paper focuses on detecting those adversarial examples by analyzing whether they come from the same distribution as the normal examples. Instead of directly training a deep neural network to detect adversarials, a much simpler approach was proposed based on statistics on outputs from convolutional layers. A cascade classifier was designed to efficiently detect adversarials. Furthermore, trained from one particular adversarial generating mechanism, the resulting classifier can successfully detect adversarials from a completely different mechanism as well. The resulting classifier is non-subdifferentiable, hence creates a difficulty for adversaries to attack by using the gradient of the classifier. After detecting adversarial examples, we show that many of them can be recovered by simply performing a small average filter on the image. Those findings should lead to more insights about the classification mechanisms in deep convolutional neural networks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “acknowledged in a growing number of recent papers in the area of computer security [6, 8, 11, 13–17] and computer vision [9, 10, 18], which focus mainly on the study of security and robustness of deep networks to adversarial inputs.”
On the (Statistical) Detection of Adversarial Examples (2017, methodology): “Research was supported in part by the Army Research Laboratory, under Cooperative Agreement Number W911NF-13-2-0045 (ARL Cyber Security CRA), and the Army Research Office under grant W911NF-13-1-0421.”
Spatially Transformed Adversarial Examples (2018, methodology): “We also test our adversarial examples against the 3 × 3 average pooling restoration mechanism (Li & Li, 2016).”
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods (2017, methodology): “We study ten detection schemes proposed in seven papers over the last year [4, 11, 12, 15, 18, 19, 24], and compare their efficacy with the other defenses in a consistent manner.”
Is Deep Learning Safe for Robot Vision? Adversarial Examples Against the iCub Humanoid (2017, methodology): “In [15], adversarial examples are detected using the output of the first convolutional layers.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Li and Li [105] used statistics of the convolution filters in CNN-based neural networks to classify the input images as clean or adversarial.”
Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification (2018, background): “One line of research [5, 6, 8, 11, 20, 24, 40] aim to detect adversarial examples, i.”
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization (2018, neutral): “, [9,10,11,12,13,14]).”
Curls & Whey: Boosting Black-Box Adversarial Attacks (2019, background): “A series of defense methods have been proposed to improve robustness of target models [15, 10, 12].”
Adversarial Machine Learning for Network Intrusion Detection Systems: A Comprehensive Survey (2023, methodology): “tion is to apply an average filter of size three by three to the input [139].”
Evaluation and Design of Robust Neural Network Defenses (2018, methodology): “[70] apply PCA to the values after inner convolutional layers of the neural network, and use a cascade classifier to detect adversarial examples.”
Towards Robust Detection of Adversarial Examples (2017, methodology): “In the adversarial setting, the strong baseline we use is the K-density estimate method (CE training + K-density detector) [8], which has shown its superiority and versatility compared to other detection-based defense methods [1, 10, 13, 21, 26] in [3].”
Adversarial Examples on Object Recognition (2020, methodology): “Similarly, Li and Li[104] developed an adversarial detector based on features extracted at every layer of a convolutional neural network.”
With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning (2018, background): “, sensitivity to model uncertainty, neuron value distribution [64, 31, 27, 37, 25].”
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance (2019, background): “…instead detect whether the given input was adversarially generated were also made (Feinman et al., 2017; Grosse et al., 2017; Metzen et al., 2017; Li & Li, 2017) but were quickly broken as well (Carlini & Wagner, 2017a), and it seems like the ’attackers’ are winning the arms-race against the…”
Intriguing properties of adversarial training (2019, background): “Meanwhile, [11, 13, 4] demonstrate it is possible to build classifiers to separate adversarial images from clean images.”