Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

By Nicolas Papernot, Patrick Drew McDaniel, Xi Wu, Somesh Jha, Ananthram Swami. IEEE Symposium on Security and Privacy, 2015

3,315 citations recorded by the source corpus.

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 1030. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.

Subfields: Artificial Intelligence, Hardware and Architecture, Molecular Biology, Safety, Robustness & Interpretability

View Paper: https://arxiv.org/abs/1511.04508

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Quoc V. Le
  5. Jianfeng Gao
  6. Bernt Schiele
  7. Sergey Levine
  8. Dacheng Tao

In Hardware and Architecture

  1. Nicholas Carlini
  2. Matthias Hein
  3. Bo Li
  4. J. Kolter
  5. Cho-Jui Hsieh
  6. Dawn Song
  7. Nicolas Papernot
  8. Ian J. Goodfellow

In Molecular Biology

  1. Nicholas Carlini
  2. Jun Zhu
  3. Nicolas Papernot
  4. Michael I. Jordan
  5. Cho-Jui Hsieh
  6. Patrick Drew McDaniel
  7. Tianyu Pang
  8. Dan Hendrycks

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Matthias Hein
  3. Nicolas Papernot
  4. Cho-Jui Hsieh
  5. Huan Zhang
  6. Florian Tramèr
  7. Aleksander Madry
  8. Bo Li

Citations in the corpus