Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
By Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra. 2016
28,183 citations recorded by the source corpus.
We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a ‘stronger’ deep network from a ‘weaker’ one even when both make identical predictions. Our code is available at https://github.com/ramprs/grad-cam/, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (http://gradcam.cloudcv.org) and a video at http://youtu.be/COjUB9Izk6E.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yoshua Bengio
- Dacheng Tao
- Jianfeng Gao
- Trevor Darrell
- Quoc V. Le
- Sergey Levine
- Percy Liang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Bernt Schiele
- Yi Yang
In Safety, Robustness & Interpretability
- Wojciech Samek
- K. Müller
- Longo Luca
- Been Kim
- Giulia Vilone
- Grgoire Montavon
- Lars Kai Hansen
- Su-In Lee
Citations in the corpus
- This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models (2020, methodology): “Further work to this end includes using GradCAM visualization [24] to quantitatively discuss face matcher behavior regarding real and fake images.”
- FILIP: Fine-grained Interactive Language-Image Pre-Training (2021, methodology): “In Figure 3, we visualize the cross-modal alignment of the proposed method for more images, in terms of both word-patch alignment as described in Section 4.5 and Grad-CAM heatmaps (Selvaraju et al., 2017).”
- Women also Snowboard: Overcoming Bias in Captioning Models (2018, methodology): “Unlike [25] who apply Grad-CAM to an entire caption, we visualize the evidence for generating specific words, i.”
- ViperGPT: Visual Inference via Python Execution for Reasoning (2023, background): “Many approaches provide explanations in the form of pixel importance, à la GradCAM [47, 65, 11, 41], some also providing textual explanations [41].”
- Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Examples are LRP [40], DeepLIFT [41], CAM [42], Grad-CAM [43], Integrated gradients [44], and SmoothGrad [45].”
- Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, methodology): “Visualizing explanations To investigate how CDEP altered a DNN’s explanations, we visualize GradCAM heatmaps (Ozbulak, 2019; Selvaraju et al., 2017) on the ISIC test dataset with a regularized and unregularized network in Fig.”
- A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “Such evaluation is performed, for example, by [33].”
- Generative Counterfactual Introspection for Explainable Deep Learning (2019, methodology): “Many CNN interpretation methods [4]–[6], [12], [13], such as GradCAM [12], utilize backpropagation to conduct sensitivity analysis by attributing the prediction to the input domain (e.g., image pixels).”