Hierarchical Attention Networks

By Paul Hongsuck Seo, Zhe L. Lin, Scott D. Cohen, Xiaohui Shen, Bohyung Han. 2016

42 citations recorded by the source corpus.

We propose a novel attention network, which accurately attends to target objects of various scales and shapes in images through multiple stages. The proposed network enables multiple layers to estimate attention in a convolutional neural network (CNN). The hierarchical attention model gradually suppresses irrelevant regions in an input image using a progressive attentive process over multiple CNN layers. The attentive process in each layer determines whether to pass or suppress feature maps for use in the next convolution. We employ local contexts to estimate attention probability at each location since it is difficult to infer accurate attention by observing a feature vector from a single location only. The experiments on synthetic and real datasets show that the proposed attention network outperforms traditional attention methods in various attribute prediction tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip H. S. Torr
2. Laura E. Barnes
3. Kamran Kowsari
4. Donald E. Brown
5. Mojtaba Heidarysafa
6. Kiana Jafari Meimandi
7. Namhoon Lee
8. Saumya Jetley

In Computer Vision and Pattern Recognition

1. Namhoon Lee
2. Saumya Jetley
3. Nicholas A. Lord
4. Philip H. S. Torr
5. Laura E. Barnes
6. Kamran Kowsari
7. Donald E. Brown
8. Mojtaba Heidarysafa

Citations in the corpus

• SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning (2016, neutral): “[22] proposed a multilayer attention networks.”
• Learn To Pay Attention (2018, methodology): “Progressive attention networks: We experiment with the progressive attention mechanism proposed by Seo et al. (2016) as part of our 2 -level attention-based VGG models.”
• EEG-Based Emotion Recognition via Channel-Wise Attention and Self Attention (2023, unknown): “multi-layer attention [39] and channel-wise attention [28].”
• Text Classification Algorithms: A Survey (2019, methodology): “N) One of the successful deep architecture for text and document classification is hierarchical attention networks (HAN). This technique was introduced by Z. Yang et al. [193] and S.P. Hongsuck et al. [194]. The structure of a HAN focuses on the document-level classification which a document has L sen”