Mitigating adversarial effects through randomization

By Cihang Xie, Zhishuai Zhang, Zhou Ren, Alan L. Yuille. 2017

1,213 citations recorded by the source corpus.

Convolutional neural networks have demonstrated high accuracy on various tasks in recent years. However, they are extremely vulnerable to adversarial examples. For example, imperceptible perturbations added to clean images can cause convolutional neural networks to fail. In this paper, we propose to utilize randomization at inference time to mitigate adversarial effects. Specifically, we use two randomization operations: random resizing, which resizes the input images to a random size, and random padding, which pads zeros around the input images in a random manner. Extensive experiments demonstrate that the proposed randomization method is very effective at defending against both single-step and iterative attacks. Our method provides the following advantages: 1) no additional training or fine-tuning, 2) very few additional computations, 3) compatible with other adversarial defense methods. By combining the proposed randomization method with an adversarially trained model, it achieves a normalized score of 0.924 (ranked No.2 among 107 defense teams) in the NIPS 2017 adversarial examples defense challenge, which is far better than using adversarial training alone with a normalized score of 0.773 (ranked No.56). The code is public available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Certified Robustness to Adversarial Examples with Differential Privacy (2018, methodology): “These defenses include model distialltion [44], automated detection of adversarial examples [22], [41], [40], application of different input transformations [27], [10], randomization [21], [11], and application of generative models [49], [25], [60].”
Adversarial Training for Free! (2019, background): “To put things further in perspective, Uesato et al. [2018] broke three defenses for (cid:15) = 2 non-targeted attacks on ImageNet [Guo et al., 2017, Liao et al., 2018, Xie et al., 2017], degrading their performance below 1%.”
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples (2018, background): “, CIFAR-10 (Krizhevsky & Hinton,2009) forMadry et al.(2018);Song et al.(2018);Ma et al.(2018); Buckman et al.(2018);Dhillon et al.(2018), and ImageNet (Krizhevsky et al.,2012) forGuo et al.(2018);Xie et al. (2018). If a defense argues security on MNIST and any other dataset, we only circumvent t”
Ensemble Adversarial Training: Attacks and Defenses (2017, background): “4% ) prepends our IRv2 adv-ens model with random padding and resizing of the input image (Xie et al., 2018).”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, methodology): “Noise can be drawn either from Gaussian [26, 24, 37], Laplace [24], Uniform [44], or Multinomial [12] distributions.”
Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, methodology): “For further improving the performance of DNN under attack, there are works attempt to introducing randomness into DNN for adversarial defense, such as randomly pruning some activation during the inference [37], randomizing the input layer [42], inserting a noise-layer right before the convolution la”
Towards Robust Neural Networks via Random Self-ensemble (2017, neutral): “[3,19,13,8,36,27,25]) against white box attacks in literature.”
Towards the first adversarially robust neural network model on MNIST (2018, background): “• Stochasticity: Certain inputs or hidden activations are shuffled or randomized (Prakash et al., 2018; Dhillon et al., 2018; Xie et al., 2018).”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “Much of the perturbation defense literature is motivated by security concerns, asserting that small, error-inducing perturbations of correctly handled examples are a “major security concern in real-world applications of DNNs” [51], “pose a great security danger to the deployment of commercial machin”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, methodology): “Original evaluation : Xie et al. (2017) propose using randomized models, so that an adversary is unable to know which model to optimize against.”
Are adversarial examples inevitable? (2018, background): “…as compression and image quilting (Guo et al., 2017), stochastic activation pruning (Dhillon et al., 2018), adding randomization at inference time (Xie et al., 2017), enhancing the conﬁdence of image labels (Song et al., 2017), and using a generative model as a defense (Samangouei et al., 2018).”
Adversarial Examples Are a Natural Consequence of Test Error in Noise (2019, background): “…2018; Das et al., 2018; 2017), Pixel Deﬂection (Prakash et al., 2018), total variance minimization (Guo et al., 2017), respresentation-guided denoising (Liao et al., 2018), and random resizing and random padding of the input image (Xie et al., 2017). failed to improve performance in Gaussian noise.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Defenses: Proposed defenses include detection and rejection methods [31, 25, 54, 60, 3, 63], pre-processing, quantization and dimensionality reduction methods [12, 73, 7], manifold-projection methods [39, 72, 82, 86], methods based on stochasticity/regularization or adapted architectures [109, 7, 68”
Deflecting Adversarial Attacks with Pixel Deflection (2018, background): “The few proposed transformation based defense techniques which work on larger-scale images are extremely recent, and currently under review [43, 15].”
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks (2019, background): “, 2018), randomization (Xie et al., 2018), input transformation (INPUT-TRANS) (Guo et al.”
Relating Adversarially Robust Generalization to Flat Minima (2021, methodology): “2 [136] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan L.”
Deep Nets: What have They Ever Done for Vision? (2018, background): “…studies showing they can be successfully attacked by imper-ceptible modiﬁcations of the images which neverthe-less cause the Deep Nets to make major mistakes for object classiﬁcation (Szegedy et al., 2014), object detection, and semantic segmentation (Xie et al., 2017) (see Figure 4 and Figure 5).”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, background): “The second approach adds additional pre-processing steps at the input in order to detect or defend against adversarial examples [11, 50, 86, 105, 111, 126, 128].”