Order-Free RNN with Visual Attention for Multi-Label Classification

By Shang-Fu Chen, Yi-Chen Chen, Chih-Kuan Yeh, Yu-Chiang Frank Wang. 2017

159 citations recorded by the source corpus.

We propose a recurrent neural network (RNN) based model for image multi-label classification. Our model uniquely integrates and learning of visual attention and Long Short Term Memory (LSTM) layers, which jointly learns the labels of interest and their co-occurrences, while the associated image regions are visually attended. Different from existing approaches utilize either model in their network architectures, training of our model does not require pre-defined label orders. Moreover, a robust inference process is introduced so that prediction errors would not propagate and thus affect the performance. Our experiments on NUS-WISE and MS-COCO datasets confirm the design of our network and its effectiveness in solving multi-label classification problems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “This has been done, for example, by using recurrent neural networks, at the expense of introducing intrinsic label ordering during training, which has been resolved by applying a category-wise max-pooling across the time dimension [62, 7, 71, 54] or by optimizing for the most likely ground truth lab”
General Multi-label Image Classification with Transformers (2020, background): “0 Order-Free RNN [6] - - - - - - - 79.”
Learning to diagnose from scratch by exploiting dependencies among labels (2017, methodology): “(2016); Chen et al. (2017) with the application of image tagging, both outside the medical context. The work of Shin et al. (2016); Wang et al. (2017) for chest x-ray annotations are closest to ours. All of them utilize out-of-the-box decoders based on recurrent neural networks (RNNs) to sequentiall”
Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification (2019, methodology): “Note that SRN and MEFF use the same ResNet101 backbone as our models, while ResNet152 is used by OF-RNN as backbone.”
The Emerging Trends of Multi-Label Learning (2020, unknown): “[154] does not require pre-defined label orders.”
Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (2018, background): “Recently, various methods [4, 5, 33, 39, 40, 43, 45] have been proposed to locate semantic regions for learning deep attentional representations.”