Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients

By Andrew Slavin Ross, Finale Doshi-Velez. 2017

754 citations recorded by the source corpus.

Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more "legitimate," interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for model predictions. We conclude by discussing this relationship between interpretability and robustness in deep neural networks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

A Style-Based Generator Architecture for Generative Adversarial Networks (2018, methodology): “FFHQ uses WGAN-GP for configuration a and nonsaturating loss [21] with R1 regularization [40, 47, 13] for configurations b–f.”
Resolution-robust Large Mask Inpainting with Fourier Convolutions (2021, methodology): “In the final loss we also use [31, 38, 7], and a discriminator-based perceptual loss or so-called feature matching loss—a perceptual loss on the features of discriminator network L DiscPL [45].”
Adversarial Training for Free! (2019, background): “Another popular defense strategy is to regularize the training loss using label smoothing, logit squeezing, or a Jacobian regularization [Shafahi et al., 2019a, Mosbach et al., 2018, Ross and Doshi-Velez, 2018, Hein and Andriushchenko, 2017, Jakubovitz and Giryes, 2018, Yu et al., 2018].”
Adversarial Vulnerability of Neural Networks Increases With Input Dimension (2018, background): “[RDV18] Andrew Slavin Ross and Finale Doshi-Velez.”
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “An alternative line of work proposes penalizing the gradients of a neural network to match human-provided binary annotations and shows the possibility to improve performance (Ross et al., 2017) and adversarial robustness (Ross & Doshi-Velez, 2018).”
Knowledge Distillation: A Survey (2020, background): “Using the natural characteristic of knowledge distillation with teacher–student architectures, knowledge distillation is used as an effective strategy to solve adversarial attacks or perturbations of deep models (Papernot et al. 2016; Ross and Doshi-Velez 2018; Goldblum et al. 2020; Gil et al. 2019)”
Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, methodology): “A similar approach was taken for softmax models by Ross & Doshi-Velez (2018).”
Robustness May Be at Odds with Accuracy (2018, background): “Ross & Doshi-Velez (2017) propose regularizing the gradient of the classiﬁer with respect to its input.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, methodology): “Ross and Doshi-Velez [52] studied input gradient regularization [165] as a method for adversarial robustness.”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, background): “While in the model interpretability community, transparency is often measured qualitatively from the perspective of a human (Kim, 2015; Krakovna and Doshi-Velez, 2016; Ross and Doshi-Velez, 2018), we note that obscurity as we define it provides one quantitative measure of the degree to which models ”
Adversarial Latent Autoencoders (2020, methodology): “In addition, we use gradient regularization techniques [8, 36, 43].”
A Benchmark for Interpretability Methods in Deep Neural Networks (2018, background): “While there is no clear way to measure “correctness”, comparing the relative merit of different estimators is often based upon human studies [31, 28, 22] which interrogate whether the ranking is meaningful to a human.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Defenses: Proposed defenses include detection and rejection methods [31, 25, 54, 60, 3, 63], pre-processing, quantization and dimensionality reduction methods [12, 73, 7], manifold-projection methods [39, 72, 82, 86], methods based on stochasticity/regularization or adapted architectures [109, 7, 68”
Learning Explainable Models Using Attribution Priors (2019, methodology): “Ross and Doshi-Velez [25] make a connection between [26] introduce the idea of training models to have correct explanations, not just good performance.”
Adversarial Weight Perturbation Helps Robust Generalization (2020, methodology): “AT explicitly flattens the input loss landscape by training on adversarially perturbed examples, while there are other methods doing this by gradient regularization [25, 41], curvature regularization [29], and local linearity regularization [39].”