Grad-CAM: Why did you say that?

By Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra. 2016

616 citations recorded by the source corpus.

We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize important regions. These localizations are combined with existing pixel-space visualizations to create a novel high-resolution and class-discriminative visualization called Guided Grad-CAM. These methods help better understand CNN-based models, including image captioning and visual question answering (VQA) models. We evaluate our visual explanations by measuring their ability to discriminate between classes, to inspire trust in humans, and their correlation with occlusion maps. Grad-CAM provides a new way to understand CNN-based models. We have released code, an online demo hosted on CloudCV, and a full version of this extended abstract.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

High Fidelity Visualization of What Your Self-Supervised Representation Knows About (2021, neutral): “Deterministic visualization methods: Many early works (Erhan et al., 2009; Zeiler & Fergus, 2014; Simonyan et al., 2013; Selvaraju et al., 2016; Smilkov et al., 2017) used gradient based techniques to visualize what is learned by neural networks.”
Sanity Checks for Saliency Maps (2018, background): “[19], GradCAM explanations correspond to the gradient of the class score (logit) with respect to the feature map of the last convolutional unit of a DNN.”
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge (2018, methodology): “An alternative is attribution methods that visualize how different parts of the input contribute to the ﬁnal output, such by generating sensitivity maps over the input (Bach et al., 2015; Baehrens et al., 2010; Selvaraju et al., 2016; Smilkov et al., 2017).”
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values (2018, background): “Introduced by Selvaraju et al. (2016), GradCAM explanations correspond to the gradient of the class score (logit) with respect to the feature map of the last convolutional unit of a DNN.”
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2017, methodology): “Saliency methods are one of the most popular local explanation methods for image classification (Erhan et al., 2009; Smilkov et al., 2017; Selvaraju et al., 2016; Sundararajan et al., 2017; Dabkowski & Gal, 2017).”
SmoothGrad: removing noise by adding noise (2017, methodology): “Similar ideas appear in (Selvaraju et al., 2016; Zhou et al., 2016), which suggest ways to combine gradients of units at multiple levels.”
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations (2017, methodology): “If the underlying model is differentiable, then one alternative is to use input gradients as local explanations [Selvaraju et al. , 2016; Simonyan et al. , 2013; Li et al. , 2015; Hechtlinger, 2016].”
Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning (2021, methodology): “These methods include Integraded Gradients (Sundararajan et al., 2017), SmoothGrad (Smilkov et al., 2017), Guided GradCAM (Selvaraju et al., 2016), etc. Motivated by gradient explanation, DeepLIFT (Shrikumar et al., 2017) propose to use slope to describes how y changes as x differs from the baseline”
Interpretable Basis Decomposition for Visual Explanation (2018, methodology): “In the first experiment, we compare explanations generated by our method head-to-head with explanations generated by Network Dissection [26] and CAM [28] and Grad-CAM [20].”
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting (2020, background): “, 2014), Guided GradCAM (Selvaraju et al., 2016), Integrated Gradients (Sundararajan et al.”
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision (2020, background): “Although deep networks are highly non-linear globally, first-order approximations have found multiple uses, for example in providing explanations [56, 59] and generating adversarial examples [26].”
An Introduction to Deep Visual Explanation (2017, background): “elation between feature maps and the weights of the ﬁnal layer. However, a drawback of this approach is that it can not be employed to networks which use multiple fully connected layers. Recently, in [10], authors proposed a relaxation generalization to CAM known as (GRAD-CAM). It computes the gradi”
Explaining Image Classifiers by Counterfactual Generation (2018, background): “Gradient-based approaches (Simonyan et al., 2013; Springenberg et al., 2014; Zhang et al., 2016; Selvaraju et al., 2016) derive a saliency map for a given input example and class target by computing the gradient of the classiﬁer output with respect to each component (e.g., pixel) of the input.”
Neural Network Attributions: A Causal Perspective (2019, background): “, 2016) or approximating the local decision boundary (around the input data point in question) via “interpretable” regressors like linear classifiers (Ribeiro et al., 2016; Selvaraju et al., 2016; Zhou & Troyanskaya, 2015; Alvarez-Melis & Jaakkola, 2017) or decision trees.”
Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases (2020, background): “There has been some research on developing a more fundamental understanding of the representations learned by deep neural networks in supervised settings [28, 29, 30, 31, 32].”