On the Connection Between Adversarial Robustness and Saliency Map Interpretability

By Christian Etmann, Sebastian Lunz, Peter Maass, Carola-Bibiane Schönlieb. 2019

181 citations recorded by the source corpus.

Recent studies on the adversarial vulnerability of neural networks have shown that models trained to be more robust to adversarial attacks exhibit more interpretable saliency maps than their non-robust counterparts. We aim to quantify this behavior by considering the alignment between input image and saliency map. We hypothesize that as the distance to the decision boundary grows,so does the alignment. This connection is strictly true in the case of linear models. We confirm these theoretical findings with experiments based on models trained with a local Lipschitz regularization and identify where the non-linear nature of neural networks weakens the relation.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Explainable machine learning in deployment (2019, background): “Model robustness improves the quality of feature importances (specifically saliency maps), confirming recent research findings [21].”
Bayes-Probe: Distribution-Guided Sampling for Prediction Level Sets (2020, background): “Such adversarial attacks produce examples that are out of distribution; indeed, a promising approach for detecting or defending against adversarial examples relies on these examples being out-of-distribution (Hendrycks & Gimpel, 2017; Etmann et al., 2019).”
Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training (2019, background): “This is less desirable as it might neglect other directions that are crucial for learning robust models [28, 17] and leads to label leaking due to high correlation between the perturbation and the decision boundary.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “[35] theoretically proved that the increase in adversarial robustness improves the alignment between input and its respective input gradient, using the case of a linear binary classifier.”
Certifiably Robust Interpretation in Deep Learning (2019, background): “[12, 13, 14, 15, 16, 17, 18, 19, 20, 21]), to the best of our knowledge, there is no practical defense for deep learning interpretation against adversarial examples [22].”