Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks

By Jörg Wagner, Jan M. Köhler, Tobias Gindele, Leon Hetzel, Thaddäus Wiedemer, Sven Behnke. 2019

132 citations recorded by the source corpus.

To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the characteristics of images, such as edges and colors. The explanations are interpretable, suited for visualizing detailed evidence and can be tested as they are valid model inputs. We qualitatively and quantitatively evaluate our approach on a multitude of models and datasets.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models (2020, background): “Our work is in a similar spirit to [18, 10, 6, 58], where meaningful perturbations have been applied to the image during model training, to improve prediction and facilitate interpretation.”
Machine Learning Interpretability: A Survey on Methods and Metrics (2019, methodology): “There are CVPR 2019 papers with a great focus on interpretability and explainability for computer vision, such as explainability methods for graph CNNs [138], interpretable and fine-grained visual explanations for CNNs [139], interpreting CNNs via Decision Trees [140], and learning to explain with c”
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks (2019, background): “Perturbation-Based approaches [3,5,6,9,10,19] work by perturbing original input and observe the change of the prediction of model.”
A Diagnostic Study of Explainability Techniques for Text Classification (2020, methodology): “Such post-hoc explanation techniques can be applicable to one speciﬁc model (Martens et al., 2008; Wagner et al., 2019) or to a broader range thereof (Ribeiro et al., 2016; Lundberg and Lee, 2017).”
Generative causal explanations of black-box classifiers (2020, methodology): “Some methods learn an inter-pretable model such as a decision tree that approximates the black-box either globally [14–16] or locally [17–20].”
EAPT: Efficient Attention Pyramid Transformer for Image Processing (2021, result): “include more feature information, and this has been observed in many previous works [23]–[25].”