Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition

By Tianshui Chen, Zhouxia Wang, Guanbin Li, Liang Lin. 2017

170 citations recorded by the source corpus.

Recognizing multiple labels of images is a fundamental but challenging task in computer vision, and remarkable progress has been attained by localizing semantic-aware image regions and predicting their labels with deep convolutional neural networks. The step of hypothesis regions (region proposals) localization in these existing multi-label image recognition pipelines, however, usually takes redundant computation cost, e.g., generating hundreds of meaningless proposals with non-discriminative information and extracting their features, and the spatial contextual dependency modeling among the localized regions are often ignored or over-simplified. To resolve these issues, this paper proposes a recurrent attention reinforcement learning framework to iteratively discover a sequence of attentional and informative regions that are related to different semantic objects and further predict label scores conditioned on these regions. Besides, our method explicitly models long-term dependencies among these attentional regions that help to capture semantic label co-occurrence and thus facilitate multi-label recognition. Extensive experiments and comparisons on two large-scale benchmarks (i.e., PASCAL VOC and MS-COCO) show that our model achieves superior performance over existing state-of-the-art methods in both performance and efficiency as well as explicitly identifying image-level semantic labels to specific object regions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, background): “Instead, they care about evaluating their topk predictions [12, 57, 7, 49, 48], by manually fixing k for all samples, or apply a fixed threshold to label probabilities [61, 5] (allowing for different number of images per sample).”
Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (2018, methodology): “RARLF(512+10crop) [5] - - - 65.”
Multi-Label Image Recognition With Graph Convolutional Networks (2019, methodology): “Following [2, 29], we use the trainval set to train our model, and evaluate the recognition performance on the test set.”
Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition (2019, background): “Some works [33, 2] further took advantage of proposal generation/visual attention mechanism to search local discriminative regions and LSTM [13] to explicitly model label dependencies.”
Deep Reasoning with Knowledge Graph for Social Relationship Understanding (2018, background): “Recently, more works are dedicated to explore message propagation by learnable neural networks like [ Chen et al. , 2018; Wang et al. , 2017 ] or neural network variants like Graph LSTM [Liang et al. , 2016; Liang et al. , 2017] and Graph CNN [Duvenaud et al. , 2015; Schlichtkrull et al. , 2017;…”
Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding (2018, methodology): “nd parts to locate discriminative regions. However, the heavy involvement of manual annotations prevents these methods from application to large-scale real-worldproblems.Recently,visualattentionmodels[5,26,30,42] were intensively proposed to automatically search the informative regions and various w”