Detecting Adversarial Samples from Artifacts

By Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, Andrew B. Gardner. 2017

984 citations recorded by the source corpus.

Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples.

Subfields: Artificial Intelligence, Genetics, Safety, Robustness & Interpretability, Anomaly & Outlier Detection

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Bernt Schiele
  2. Percy Liang
  3. Yoshua Bengio
  4. Nicholas Carlini
  5. Quoc V. Le
  6. Mohit Bansal
  7. Sergey Levine
  8. Jun Zhu

In Genetics

  1. Fabio Roli
  2. Battista Biggio
  3. Giorgio Fumera
  4. Nicholas Carlini
  5. Ajmal Saeed Mian
  6. Naveed Akhtar
  7. Matthias Hein
  8. Dawn Song

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Matthias Hein
  3. Cho-Jui Hsieh
  4. Nicolas Papernot
  5. Bo Li
  6. Huan Zhang
  7. Dawn Song
  8. Bernt Schiele

In Anomaly & Outlier Detection

  1. Nicholas Carlini
  2. Nicolas Papernot
  3. Patrick Drew McDaniel
  4. Matthias Hein
  5. Dawn Song
  6. Lewis Smith
  7. Ling Shao
  8. Michael Backes

Citations in the corpus