Multilabel Image Classification With Regional Latent Semantic Dependencies

By Junjie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu. 2016

183 citations recorded by the source corpus.

Deep convolution neural networks (CNNs) have demonstrated advanced performance on single-label image classification, and various progress also has been made to apply CNN methods on multilabel image classification, which requires annotating objects, attributes, scene categories, etc., in a single shot. Recent state-of-the-art approaches to the multilabel image classification exploit the label dependencies in an image, at the global level, largely improving the labeling capacity. However, predicting small objects and visual concepts is still challenging due to the limited discrimination of the global visual features. In this paper, we propose a regional latent semantic dependencies model (RLSD) to address this problem. The utilized model includes a fully convolutional localization architecture to localize the regions that may contain multiple highly dependent labels. The localized regions are further sent to the recurrent neural networks to characterize the latent semantic dependencies at the regional level. Experimental results on several benchmark datasets show that our proposed model achieves the best performance compared to the state-of-the-art models, especially for predicting small objects occurring in the images. Also, we set up an upper bound model (RLSD+ft-RPN) using bounding-box coordinates during training, and the experimental results also show that our RLSD can approach the upper bound without using the bounding-box annotations, which is more realistic in the real world.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “2018 [71] VGG 3 82081 n/a 40137 n/a 66.”
ELASTIC: Improving CNNs With Dynamic Scaling Policies (2018, methodology): “Following the literature of multilabel classification[41, 7, 39, 17], results are evaluated using macro/micro evaluations.”
Multi-label Image Recognition by Recurrently Discovering Attentional Regions (2017, methodology): “We compare the performance of our proposed method against the following state-of-the-art approaches: FeV+LV-20-VD [30], HCP [27], RLSD [31], CNN-RNN [26], VeryDeep [24] and CNN-SVM [23] on the VOC 2007 dataset.”
Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers (2019, unknown): “…of the proposed method against the following state-of-the-art approaches: HCP-2000 [Wei et al., 2014], HCP-VGG [Wei et al., 2016], RLSD+ft-RPN [Zhang et al., 2018], CNN+RNN [Wang et al., 2016], CNN+LSTM [Wang et al., 2017], PF-DLDL [Gao et al., 2017a] and the method of [Zhou et al., 2018].”
Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN (2020, unknown): “We compare the performance of the proposed method against the following typical multi-label approaches: ‘HCP-2000’ [29], ‘HCP-VGG’ [30], ‘RLSD+ftRPN’ [40] and ‘KA-ResNet’.”
Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification (2019, methodology): “To further evaluate our methods, we compares DecoupleNet with several state-of-the-art approaches, such as WARP [9], RLSD [22], CNN-RNN [4] and OF-RNN [6].”
Attend and Imagine: Multi-Label Image Classification With Visual Attention and Recurrent Neural Networks (2019, methodology): “CNN can be directly applied on the multi-label image classification problem, such as [14]–[17], [26].”
Multi-Label Image Recognition With Graph Convolutional Networks (2019, unknown): “0 RLSD [34] 96.”
Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (2018, methodology): “MCG-CNN-LSTM [43] 64.”