Learn To Pay Attention

By Saumya Jetley, Nicholas A. Lord, Namhoon Lee, Philip H. S. Torr. International Conference on Learning Representations, 2018

474 citations recorded by the source corpus.

We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification. The module takes as input the 2D feature vector maps which form the intermediate representations of the input image at different stages in the CNN pipeline, and outputs a 2D matrix of scores for each map. Standard CNN architectures are modified through the incorporation of this module, and trained under the constraint that a convex combination of the intermediate 2D feature vectors, as parameterised by the score matrices, must \textit{alone} be used for classification. Incentivised to amplify the relevant and suppress the irrelevant or misleading, the scores thus assume the role of attention values. Our experimental observations provide clear evidence to this effect: the learned attention maps neatly highlight the regions of interest while suppressing background clutter. Consequently, the proposed function is able to bootstrap standard CNN architectures for the task of image classification, demonstrating superior generalisation over 6 unseen benchmark datasets. When binarised, our attention maps outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation as demonstrated on the Object Discovery dataset. We also demonstrate improved robustness against the fast gradient sign method of adversarial attack.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c70218603f0af1be5d063056cbe629e042141a86

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Knowledge Squeezed Adversarial Network Compression (2019, methodology): “Different from them, we propose to squeeze the intermediate knowledge by using a taskdriven attention [46], such that the compact knowledge from teacher network could accommodate the size of student network.”
Attention U-Net: Learning Where to Look for the Pancreas (2018, methodology): “The contributions of this work can be summarised as follows: • We take the attention approach proposed in [11] a step further by proposing grid-based gating that allows attention coefﬁcients to be more speciﬁc to local regions.”
A Comprehensive Survey of Scene Graphs: Generation and Application (2021, background): “mechanism [56] and deep supervision [57], to propose a multimodal attention translation embedding method.”
Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning (2019, background): “The attention technique strengthens key objects, while suppresses the background [46, 4, 17].”
An Attentive Survey of Attention Models (2019, background): “2018; Shuai Yu 2019] as well as RNN-based sequential models [Kang and McAuley 2018; Qiannan et al. 2019; Ying et al. 2018b; Zhou et al. 2018]. Learning effective user and item representations lies at the heart of recommender systems. Consequently, Zhu et al. [2020] use attention for cross domain rec”
Unsupervised Attention-guided Image to Image Translation (2018, background): “More recently, attention has been enforced on activation functions to select only task-relevant features [2, 21].”
Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization (2019, background): “layed an important role in deep learning to mimic human visual mechanism. In [49], the attention is used to make sure the student model focuses on the discriminative regions as teacher model does. In [21], the cascade attention mechanism is proposed to guide the different layers of CNN and concatena”
Visual Attention Methods in Deep Learning: An In-Depth Survey (2022, methodology): “tiple computer vision tasks, either employing a different attention map for every image pixel, comparing it with the representations of other pixels [3], [4], [9] or generating an attention map to extract the global representation for the whole image [10], [11].”
Deep Learning for Cardiac Image Segmentation: A Review (2019, background): “• Attention units (69, 70, 76), which learn to adaptively recalibrate features spatially (see Figure 8B); • Squeeze-and-excitation blocks (77), which are used to recalibrate features with learnable weights across channels;”
See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks (2019, background): “Differentiable attentions, which are inspired by human perception [13, 58], have been widely studied in deep neural networks [26, 56, 38, 23, 57, 62, 15].”
Embedding Human Knowledge into Deep Neural Network via Attention Map (2019, unknown): “…vision ﬁeld (Ribeiro et al., 2016; Chattopadhay et al., 2018; Ramprasaath et al., 2017; Smilkov et al., 2017; Zeiler and Fergus, 2014; Zhou et al., 2016; Fukui et al., 2019; Montavon et al., 2018; Springenberg et al., 2015; Fong and Vedaldi, 2017; Petsiuk et al., 2018; Jetley et al., 2018).”
A Survey on Vision Transformer (2020, methodology): “Trainable attention for classification consists of two main streams: hard attention [3, 87, 134] regarding the use of an image region and soft attention [125, 60, 43, 102] generating non-rigid feature maps.”
Recent advances and clinical applications of deep learning in medical image analysis (2021, background): “…U-Net based segmentation approaches also benefit from adversarial training (Xue et al., 2018; Zhang et al., 2020b), attention mechanism (Jetley et al., 2018; Anderson et al., 2018; Oktay et al., 2018; Nie et al., 2018; Sinha and Dolz, 2021), and uncertainty estimation (Wang et al.,…”
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection (2018, background): “Very recently, end-to-end trainable attention-based methods have been proposed to improve the performance of action recognition [9] or image classiﬁcation [20].”
Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images (2018, unknown): “In computer vision, attention mechanisms are applied to a variety of problems, including image classification (Jetley et al., 2018; Wang et al., 2017a; Zhao et al., 2017), segmentation (Ren and Zemel, 2016), action recognition (Liu et al., 2017; Pei et al., 2016; Wang et al., 2017b), image…”