Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks

By Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr (Peter) Mardziel, Xia Hu. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019

1,511 citations recorded by the source corpus.

Recently, increasing attention has been drawn to the internal mechanisms of convolutional neural networks, and the reason why the network makes specific decisions. In this paper, we develop a novel post-hoc visual explanation method called Score-CAM based on class activation mapping. Unlike previous class activation mapping based approaches, Score-CAM gets rid of the dependence on gradients by obtaining the weight of each activation map through its forward passing score on target class, the final result is obtained by a linear combination of weights and activation maps. We demonstrate that Score-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks, it also passes the sanity check. We also indicate its application as debugging tools. The implementation is available1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2e257b5f0a0148076e9161202880b022e521ac59

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization (2020, positive): “Finally, score class activation mapping (Score-CAM) based visualization technique was used to demonstrate the regions of the X-ray images that contribute in the classification by the CNNs to confirm whether the segmented lungs X-ray images based classification is more reliable than that of whole X-r”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, neutral): “visualization [245], feature map visualization [246], [247],”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “ScoreCAM [127] also uses gradient information but assigns importance to each activation map by the notion of Increase of Confidence.”
Decoupling Deep Learning for Interpretable Image Recognition (2022, methodology): “Saliency maps [1, 33, 27, 2, 31, 23] use localization as an explanation for predictions, but this only provides the network’s area of interest for a given image, which does not fully represent the way the network makes its decisions [22].”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “We ﬁnd the localization performance of LayerCAM outperforms Grad-CAM [2], Grad-CAM++ [3], ScoreCAM [33], and NormGrad [36] by a large margin, especially in shallow layers.”
Automatic and Reliable Leaf Disease Detection Using Deep Learning Techniques (2021, methodology): “Among the numerous visualization techniques available, such as SmoothGrad [32], Grad-CAM [33], Grad-CAM++ [34], and Score-CAM [35], the recently proposed Score-CAM was used in this study due to its promising output.”