Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural Networks

By Chunshui Cao, Xianming Liu, Yi Yang, Yinan Yu, Jiang Wang, Zilei Wang, Yongzhen Huang, Liang Wang, Chang Huang, Wei Xu and 2 more. 2015

398 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Recent advances in convolutional neural networks (2015, background): “Speciﬁcally, stochastic pooling ﬁrst computes the probabilities p for each region R j by normalizing the activations within the region, i.e ., p i = a i / k j ( a k ) .”
Real Time Image Saliency for Black Box Classifiers (2017, methodology): “Center Grad [12] Guid [13] LRP [1] CAM [20] Exc [18] Feed [2] Mask [3] [3] which recalculated originally reported results using few different mask thresholding techniques and achieved slightly lower error rates.”
Residual Attention Network for Image Classification (2017, background): “However, a new process, reinforcement learning [30] or optimization [2] is involved during the training step.”
On the Binding Problem in Artificial Neural Networks (2020, background): “by highlighting locations that are (maximally) informative with respect to a discriminative task (Gao and Vasconcelos, 2005; Cao et al., 2015; Zhmoginov et al., 2019).”
Top-Down Neural Attention by Excitation Backprop (2016, methodology): “We follow the protocol of Feedback Net [7] for a fair comparison.”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, methodology): “ision for visual question answering (Chen et al., 2015), image classiﬁcation (Xiao et al., 2015), caption generation (Xu et al., 2015), image segmentation (Hong et al., 2016) and object localization (Cao et al., 2015). Mnih et al. (2014) apply attention in recurrent neural networks (RNNs) to extract”
Image Super-Resolution Using Very Deep Residual Channel Attention Networks (2018, background): “Recently, tentative works have been proposed to apply attention into deep neural networks [23, 26, 27], ranging from localization and understanding in images [28, 29] to sequence-based networks [30, 31].”
RISE: Randomized Input Sampling for Explanation of Black-box Models (2018, background): “Alternatively, ﬁrst-person models [2, 8, 23, 33] aim to generate explanations providing evidence for the model’s underlying decision process without using an additional model.”
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution (2020, background): “Inspired by the human vision system, the mechanism of looking and thinking twice has been instantiated in computer vision, and demonstrated outstanding performance [5,6,58].”
Exploiting Saliency for Object Segmentation from Image Level Labels (2017, methodology): “Starting from simple strategies such as “probing classifier with different image areas occluded” [50], or back-propagating the class score gradient on the image [41]; significantly more involved strategies have been proposed, mainly by modifying the back-propagation strategy [43, 51, 40], or by solv”
Rotate to Attend: Convolutional Triplet Attention Module (2020, background): “ure rich contextual information using intersecting strips. Xiao et al. [36] propose a pipeline integrated with one bottom-up and two top-down attention for ﬁne grained image classiﬁcation. Cao et al. [1] introduce the ’Look and Think Twice’ mechanism which is based on a computational feedback proces”
Recurrent Mixture Density Network for Spatiotemporal Visual Attention (2016, methodology): “In particular, deep neural networks have been trained to produce saliency maps in this way [41, 14, 42, 32, 43, 44].”
Multi-attention Network for One Shot Learning (2017, background): “Attention models have been applied to a variety of computer vision problems including image classification [25, 1, 8], semantic segmentation [3], visual tracking [5], person identification [9], image captioning [4, 6, 26] and question answering [27].”
Squeeze-and-Excitation Networks (2017, background): “The benefits of such a mechanism have been shown across a range of tasks, from localisation and understanding in images [3, 19] to sequence-based models [2, 28].”
Multi-context Attention for Human Pose Estimation (2017, background): “Visual Attention Mechanism Since the visual attention model is computationally efficient and is effective in understanding images, it has achieved great success in various tasks such as machine translation [3], object recognition [2, 17, 6, 40], image captioning [47, 41], image question answering [4”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, background): “Based on the deep responses, top-down attention meth-ods are proposed to generate reﬁned class saliency maps by exploring visual attention evidence [4, 43].”