MagNet: A Two-Pronged Defense against Adversarial Examples

By Dongyu Meng, Hao Chen. 2017

1,318 citations recorded by the source corpus.

Deep learning has shown impressive performance on hard perceptual problems. However, researchers found deep learning systems to be vulnerable to small, specially crafted perturbations that are imperceptible to humans. Such perturbations cause deep learning systems to mis-classify adversarial examples, with potentially disastrous consequences where safety or security is crucial. Prior defenses against adversarial examples either targeted specific attacks or were shown to be ineffective. We propose MagNet, a framework for defending neural network classifiers against adversarial examples. MagNet neither modifies the protected classifier nor requires knowledge of the process for generating adversarial examples. MagNet includes one or more separate detector networks and a reformer network. The detector networks learn to differentiate between normal and adversarial examples by approximating the manifold of normal examples. Since they assume no specific process for generating adversarial examples, they generalize well. The reformer network moves adversarial examples towards the manifold of normal examples, which is effective for correctly classifying adversarial examples with small perturbation. We discuss the intrinsic difficulties in defending against whitebox attack and propose a mechanism to defend against graybox attack. Inspired by the use of randomness in cryptography, we use diversity to strengthen MagNet. We show empirically that MagNet is effective against the most advanced state-of-the-art attacks in blackbox and graybox scenarios without sacrificing false positive rate on normal examples.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Certified Robustness to Adversarial Examples with Differential Privacy (2018, methodology): “These defenses include model distialltion [44], automated detection of adversarial examples [22], [41], [40], application of different input transformations [27], [10], randomization [21], [11], and application of generative models [49], [25], [60].”
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “The latter approach has been ﬁrstly proposed in 2008 [43] as an e ﬀ ective way to hide information about the classiﬁcation function to the attacker, with recent follow-ups in [7, 45] to counter adversarial examples.”
Adversarial Training for Free! (2019, background): “Alternatively, robustness may be achieved by detecting and rejecting adversarial examples [Ma et al., 2018, Meng and Chen, 2017, Xu et al., 2017].”
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks (2019, background): “A number of defenses have been proposed [43], [44], [45], [46], [47], yet many have shown to be less effective against an adaptive adversary [48], [49], [50], [51].”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, background): “Magnet [24] trains a classiﬁer on manifolds of normal examples to detect adversarial perturbations without prior knowledge of the attack.”
Attacking Graph-based Classification via Manipulating the Graph Structure (2019, neutral): “[13,27, 41,47]) and data poisoning attacks (e.g., [5,32,54]).”
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks (2017, methodology): “We compare our results with MagNet [24] in Table 6.”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “Since the initial discovery of adversarial examples, a wealth of non randomized defense approaches have also been proposed, inspired by various machine learning domains such as adversarial training [18, 27], image reconstruction [29, 39] or robust learning [18, 27].”
Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models (2018, methodology): “These defenses can be grouped under three different approaches: (1) modifying the training data to make the clas-siﬁer more robust against attacks, e.g., adversarial training which augments the training data of the classiﬁer with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015), ”
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach (2018, methodology): “In addition, detection methods such as feature squeezing (Xu et al., 2017) and example reforming (Meng & Chen, 2017) can also be used to identify adversarial examples.”
The Robust Manifold Defense: Adversarial Training using Generative Models (2017, methodology): “Several methods train neural networks to project an image on the manifold [33, 42] but these are end-to-end diﬀerentiable and hence easy to attack [9].”
Towards Robust Neural Networks via Random Self-ensemble (2017, methodology): “In addition to changing the network structure, there are other methods [26, 15, 6, 8] “detecting” the adversarial examples, which are not in the scope of our paper.”
Mitigating adversarial effects through randomization (2017, background): “Recent works (Papernot et al., 2016b; Kurakin et al., 2017; Tram`er et al., 2017; Metzen et al., 2017; Feinman et al., 2017; Meng & Chen, 2017) are making progress on this line of research.”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Progress has been made by discretizing the input space [71], training on adversarial examples [72], [18] or attempting to remove the adversarial perturbation [73], [74].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, methodology): “Different from gradient-based adversarial generating approaches, Chen et al. proposed a Zeroth Order Optimization (ZOO) based attack [73].”
Towards the first adversarially robust neural network model on MNIST (2018, background): “Other ideas in similar directions include the use of denoising autoencoders in (Gu & Rigazio, 2014) and (Liao et al., 2017) as well as MagNets (Meng & Chen, 2017) (which projects or rejects inputs depending on their distance to the data manifold).”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “Because many perturbation defense papers we reviewed apply defenses that increase error on the test set [4, 7, 9, 14, 20, 23, 36, 42, 45, 103, 104], these methods are actually more vulnerable to the test set attack than an undefended model.”