Top-Down Neural Attention by Excitation Backprop

By Jianming Zhang, Sarah Adel Bargal, Zhe L. Lin, Jonathan Brandt, Xiaohui Shen, Stan Sclaroff. 2016

1,058 citations recorded by the source corpus.

We aim to model the top-down attention of a convolutional neural network (CNN) classifier for generating task-specific attention maps. Inspired by a top-down human visual attention model, we propose a new backpropagation scheme, called Excitation Backprop, to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process. Furthermore, we introduce the concept of contrastive attention to make the top-down attention maps more discriminative. We show a theoretic connection between the proposed contrastive attention formulation and the Class Activation Map computation. Efficient implementation of Excitation Backprop for common neural network layers is also presented. In experiments, we visualize the evidence of a model’s classification decision by computing the proposed top-down attention maps. For quantitative evaluation, we report the accuracy of our method in weakly supervised localization tasks on the MS COCO, PASCAL VOC07 and ImageNet datasets. The usefulness of our method is further validated in the text-to-region association task. On the Flickr30k Entities dataset, we achieve promising performance in phrase localization by leveraging the top-down attention of a CNN model that has been trained on weakly labeled web images. Finally, we demonstrate applications of our method in model interpretation and data annotation assistance for facial expression analysis and medical imaging tasks.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Women also Snowboard: Overcoming Bias in Captioning Models (2018, methodology): “Finally, to measure if our models are “right for the right reasons” we consider the pointing game [37] evaluation on the MSCOCOBalanced dataset.”
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2015, methodology): “rately than our linear CCA embedding of Section 4. Wang et al. (2016b) formulate a linear program to localize all the phrases from a caption jointly, taking their semantic relationships into account. Zhang et al. (2016) perform phrase localization with a tag prediction network and a top-down attenti”
Visual Entailment: A Novel Task for Fine-Grained Image Understanding (2019, background): “Recent work addresses this requirement either through saliency-map visualizations [61, 77, 49], attention mechanism [75, 79, 51, 14], or other analysis [30, 39, 56, 58].”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, background): “[46] introduce contrastive Marginal Winning Probability (c-MWP), a probabilistic Winner-Take-All formulation for modelling the top-down attention for neural classification models which can highlight discriminative regions.”
Recent advances in convolutional neural networks (2015, background): “Speciﬁcally, stochastic pooling ﬁrst computes the probabilities p for each region R j by normalizing the activations within the region, i.e ., p i = a i / k j ( a k ) .”
Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks (2019, methodology): “[17], c) Gradient [38], d) Guided Backprop [42] , e) Contrastive Excitation Backprop [49], f) Grad-CAM [35], g) Occlusion [48], h) FGVis (ours).”
Learning to Explain With Complemental Examples (2018, background): “The primary stream is visualizing where the classifier weighs for its prediction by assigning an importance to each element in the input space, by propagating the prediction to the input space [24, 3, 29, 23, 30, 8, 31], or by learning the instance-wise importance of elements [4, 5, 15] with an auxi”
Visual Feature Attribution Using Wasserstein GANs (2017, neutral): “Examples include Guided Backprop [53], Excitation Backprop [62], Integrated Gradients [54], meaningful perturbations [13].”
Methods for interpreting and understanding deep neural networks (2017, methodology): “This modular approach was also used by Zhang et al. [56] to implement the excitation backprop method.”
Interpretable Explanations of Black Boxes by Meaningful Perturbation (2017, methodology): “…however, our method performs best with value thresholding and is competitive when using the other methods: It beats gradient [15] and guided backprop [16] when using energy thresholding; beats LRP [1], CAM [22], and contrastive excitation backprop [20] when using mean thresholding (recall from ﬁg.”
RUDDER: Return Decomposition for Delayed Rewards (2018, methodology): “ntroduced to interpret machine learning models. LRP is an extension of the contribution-propagation algorithm [71] based on the contribution approach [94]. Recently “excitation backprop” was proposed [147], which is like LPR but uses only positive weights and shifts the activation function to have n”
Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning (2019, background): “They are frequently used to explain how deep networks classify images in computer vision applications (Simonyan et al., 2014; Zeiler & Fergus, 2014; Springenberg et al., 2015; Ribeiro et al., 2016; Dabkowski & Gal, 2017; Fong & Vedaldi, 2017; Selvaraju et al., 2017; Shrikumar et al., 2017; Smilkov e”
Explainable machine learning in deployment (2019, methodology): “Organization B has tried feature visualization and activation visualization techniques that get attributions by back-propagating gradients to regions of interest [5, 67].”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “Decomposition methods can be subdivided in three categories: (1) structure approaches, where the function itself has the particular summation structure, and where individual pixel-wise contributions can be identified as the summands of the function [131], (2) analytic approaches, where a local analy”
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting (2020, methodology): “We use the Pointing Game experiment (PG) (Zhang et al., 2018) for this evaluation, which was introduced to measure the discriminative-ness of a visualization method for target object localization.”
Learning to Continually Learn (2020, background): “n of existing knowledge. Other approaches similarly seek to modulate learning via task-speciﬁc synaptic importance [45], via the top-down attention mechanism of contrastive excitation backpropagation [20, 46], by interleaving pseudo examples generated from the current weights to limit plasticity [25”
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded (2019, methodology): “In addition to these diagnostic datasets, another powerful class of tools for observing this shortcoming has been gradient-based explanation techniques [27, 35, 26, 24] which allow researchers to examine which portions of the Figure 1: Our approach, HINT, aligns visual explanations for output decisi”