Recurrent Attentional Networks for Saliency Detection

By Jason Kuen, Zhenhua Wang, G. Wang. 2016

243 citations recorded by the source corpus.

Convolutional-deconvolution networks can be adopted to perform end-to-end saliency detection. But, they do not work well with objects of multiple scales. To overcome such a limitation, in this work, we propose a recurrent attentional convolutional-deconvolution network (RACDNN). Using spatial transformer and recurrent network units, RACDNN is able to iteratively attend to selected image sub-regions to perform saliency refinement progressively. Besides tackling the scale problem, RACDNN can also learn context-aware features from past iterations to enhance saliency refinement in future iterations. Experiments on several challenging saliency detection datasets validate the effectiveness of RACDNN, and show that RACDNN outperforms state-of-the-art saliency detection methods.

Subfields: Computer Vision and Pattern Recognition, Sensory Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Sensory Systems

Citations in the corpus

Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points (2018, neutral): “Similar hard attention was used in multiple object recognition [2], object localization [7, 38, 23], saliency map generation [27], and action detection [60].”
Pose-conditioned Spatio-Temporal Attention for Human Action Recognition (2017, methodology): “In [19], a hard attention model generates saliency maps.”
Multi-attention Network for One Shot Learning (2017, methodology): “Our method is also related to saliency detection [11].”
Reverse Attention for Salient Object Detection (2018, methodology): “[28] applied recurrent unit into FCNs to iteratively refine each salient region.”
PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection (2017, methodology): “In [17], authors use a recurrent attention model to select local regions to refine their saliency maps.”
Convolutional Neural Networks with Alternately Updated Clique (2018, background): “There are also some studies that design attention mechanism tied with recurrent neural networks [28, 24, 8].”
Salient Object Detection in the Deep Learning Era: An In-Depth Survey (2019, methodology): “• RACDNN [62] produces a coarse saliency map using an encoder-decoder stream, and progressively reﬁnes different local object regions.”
Learning to Detect Salient Objects with Image-Level Supervision (2017, methodology): “Early works [46, 57, 26, 4] use DNNs in a patch-by-patch scanning manner, leading to numerous redundant computations.”
CRAM: Clued Recurrent Attention Model (2018, methodology): “Encoder is similar to RACDNN[7] with modiﬁed spatial transformer network[6] and inserted clue.”
Human Action Recognition: Pose-Based Attention Draws Focus to Hands (2017, methodology): “In [20], a hard attention model generates saliency maps.”
Object Detection With Deep Learning: A Review (2018, background): “proposed a recurrent attentional convolutional-deconvolution network (RACDNN) with several spatial transformer and recurrent network units to conquer this problem [148].”
A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation (2020, background): “[4], [49], image classification [50], [51], [52], [53], action recognition [54], saliency detection [55], and so on.”
An Empirical Study of Language CNN for Image Captioning (2016, methodology): “Models based on RNNs have dominated recent sequence modeling tasks [23, 31, 32, 44], and most of the recent image captioning models are based on LSTM networks [6, 19, 34].”
Salient Object Detection With Pyramid Attention and Salient Edges (2019, unknown): “Other methods leverage global and local saliency information [21, 34, 54, 29], combine pixel- and segment-level features [22], inspire connections between network layers [14], or explore more complex deep architectures [18, 26, 55, 37, 32].”
Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency (2019, methodology): “To address the feature issue, deep saliency models [35], [36], [37] proposed to use feature representations learned from data by using Convolutional Neural Networks (CNNs).”
Progressive Attention Guided Recurrent Network for Salient Object Detection (2018, background): “[11] also propose a recurrent based network, that is, recurrent attentional convolutional-deconvolution network (RACDNN).”
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection (2018, methodology): “Recently, significant progress has been made by deep learning based methods [26, 36, 17, 28, 15, 16, 18, 22, 10, 29, 33, 3], which can be broadly categorized into region-based and Fully Convolutional Network (FCN)-based methods.”
Differentiable Patch Selection for Image Recognition (2021, methodology): “These have been applied sequentially to extract several regions of interest [13, 24] as part of recurrent neural networks.”
Deep Level Sets for Salient Object Detection (2017, methodology): “Kuen et al [25] utilize a convolutional network to generate a coarse map and then refine it with an attentional recurrent network.”