Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization
By Shir Gur, Ameen Ali, Lior Wolf. 2020
43 citations recorded by the source corpus.
Neural network visualization techniques mark image locations by their relevancy to the network's classification. Existing methods are effective in highlighting the regions that affect the resulting classification the most. However, as we show, these methods are limited in their ability to identify the support for alternative classifications, an effect we name the saliency bias hypothesis. In this work, we integrate two lines of research: gradient-based methods and attribution-based methods, and develop an algorithm that provides per-class explainability. The algorithm back-projects the per pixel local influence, in a manner that is guided by the local attributions, while correcting for salient features that would otherwise bias the explanation. In an extensive battery of experiments, we demonstrate the ability of our methods to class-specific visualization, and not just the predicted label. Remarkably, the method obtains state of the art results in benchmarks that are commonly applied to gradient-based methods as well as in those that are employed mostly for evaluating attribution methods. Using a new unsupervised procedure, our method is also successful in demonstrating that self-supervised methods learn semantic information. Our code is available at: https://github.com/shirgur/AGFVisualization.
Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Andrew Zisserman
- Trevor Darrell
- Andrea Vedaldi
- Ming-Hsuan Yang
- Cordelia Schmid
- Xiaogang Wang
In Safety, Robustness & Interpretability
- Hila Chefer
- Julien Mairal
- Lior Wolf
- Shir Gur
- Herv'e J'egou
- Ishan Misra
- Armand Joulin
- Piotr Bojanowski
Citations in the corpus
- Emerging Properties in Self-Supervised Vision Transformers (2021, methodology): “Note that self-supervised convnets also contain information about segmentations but it requires dedicated methods to extract it from their weights [31].”
- Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers (2021, background): “Since LRP and most variants [25, 33, 22] are class agnostic [16], class-specific extensions were introduced [13, 16, 14].”
- Transformer Interpretability Beyond Attention Visualization (2020, methodology): “Other variants of attribution methods include RAP [28], AGF [17], DeepLIFT [33], and DeepSHAP [24].”