Defensive Distillation is Not Robust to Adversarial Examples

By Nicholas Carlini, David A. Wagner. 2016

353 citations recorded by the source corpus.

We show that defensive distillation is not secure: it is no more resistant to targeted misclassification attacks than unprotected neural networks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “…new defenses (Lu et al., 2017; Papernot et al., 2017; Metzen et al., 2017; Hendrycks & Gimpel, 2017a) often quickly succumb to new attacks (Evtimov et al., 2017; Carlini & Wagner, 2017; 2016), with some exceptions for perturbations on small images (Schott et al., 2018; Madry et al., 2018).”
Towards Deep Learning Models Resistant to Adversarial Attacks (2017, background): “Due to the growing body of work on adversarial examples [8, 6, 23, 17, 3, 25, 7, 12], we focus only on the most related papers here.”
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks (2019, background): “A number of defenses have been proposed [43], [44], [45], [46], [47], yet many have shown to be less effective against an adaptive adversary [48], [49], [50], [51].”
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks (2018, background): “Such a system is expected to be part of any partially- or fully-autonomous s lf-driving car [9].”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “[4] Nicholas Carlini and David Wagner.”
Certified Defenses against Adversarial Examples (2018, methodology): “Some attacks (Carlini & Wagner, 2016; Tramèr et al., 2017) evade these defenses and witness a large f ( A opt ( x )) .”
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles (2017, methodology): “We used the following weak and strong attacks [1], with various choices of hyper-parameters, to test the robustness of two neural networks with different architectures.”
Efficient Defenses Against Adversarial Attacks (2017, methodology): “Among these attack-agnostic techniques, we find defensive distillation [5, 33, 35], which hardens the model in two steps: first, a classification model is trained and its softmax layer is smoothed by division with a constant T ; then, a second model is trained using the same inputs, but instead of f”
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly (2017, methodology): “Our experiments demonstrate that there is a trade-off between classification confidence and detection easiness for adversarial examples.”
Adversarially Robust Generalization Requires More Data (2018, background): “[10] Nicholas Carlini and David Wagner.”
One Pixel Attack for Fooling Deep Neural Networks (2017, background): “However, recent studies show that many of these defense and detection methods can be effectively evaded by conducting little modification on the original attacks [44], [45], [59].”
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients (2017, background): “It has been shown to be an effective defense against white-box FGSM attacks, but Carlini and Wagner (2016) have shown that it is not robust to all kinds of attacks.”
On Evaluating Adversarial Robustness (2019, background): “At the same time, even relatively simple and efficient attacks (DeepFool and JSMA) (Moosavi-Dezfooli et al., 2016; Papernot et al., 2016b) can still be useful for evaluating adversarial robustness if applied correctly (Jetley et al., 2018; Carlini & Wagner, 2016).”
Evaluating and Understanding the Robustness of Adversarial Logit Pairing (2018, background): “Unfortunately, many proposed defenses have had their claims invalidated by new attacks within their corresponding threat models [4, 8, 5, 6, 2, 12, 1].”
Are adversarial examples inevitable? (2018, background): “More sophisticated defenses that rely on network distillation (Papernot et al., 2016b) and specialized activation functions (Zantedeschi et al., 2017) were also toppled by strong attacks (Papernot et al., 2016a; Tram ` er et al., 2017b; Carlini & Wagner, 2016; 2017a).”