Knowledge Guided Disambiguation for Large-Scale Scene Classification With Multi-Resolution CNNs

By Limin Wang, Sheng Guo, Weilin Huang, Yuanjun Xiong, Yu Qiao. 2016

158 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have made remarkable progress on scene recognition, partially due to these recent large-scale scene datasets, such as the Places and Places2. Scene categories are often defined by multi-level information, including local objects, global layout, and background environment, thus leading to large intra-class variations. In addition, with the increasing number of scene categories, label ambiguity has become another crucial issue in large-scale classification. This paper focuses on large-scale scene recognition and makes two major contributions to tackle these issues. First, we propose a multi-resolution CNN architecture that captures visual content and structure at multiple levels. The multi-resolution CNNs are composed of coarse resolution CNNs and fine resolution CNNs, which are complementary to each other. Second, we design two knowledge guided disambiguation techniques to deal with the problem of label ambiguity: 1) we exploit the knowledge from the confusion matrix computed on validation data to merge ambiguous classes into a super category and 2) we utilize the knowledge of extra networks to produce a soft label for each image. Then, the super categories or soft labels are employed to guide CNN training on the Places2. We conduct extensive experiments on three large-scale image datasets (ImageNet, Places, and Places2), demonstrating the effectiveness of our approach. Furthermore, our method takes part in two major scene recognition challenges, and achieves the second place at the Places2 challenge in ILSVRC 2015, and the first place at the LSUN challenge in CVPR 2016. Finally, we directly test the learned representations on other scene benchmarks, and obtain the new state-of-the-art results on the MIT Indoor67 (86.7%) and SUN397 (72.0%). We release the code and models at https://github.com/wanglimin/MRCNN-Scene-Recognition.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Visual Dialog (2016, background): “tion [55], object detection [27] – to ‘high-level’ AI tasks such as learning to play Atari video games [35] and Go [47], answering reading comprehension questions by understanding short stories [15, 57], and even answering questions about images [4, 32, 41] and videos [49, 50]!”
Temporal Segment Networks for Action Recognition in Videos (2017, background): “Recently, Convolutional Neural Networks (Con-vNets) [6] have achieved great success in classifying images of objects [7], [8], [9], scenes [10], [11], [12], and complex events [13], [14].”
Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification (2019, background): “Great successes have been achieved in image classification, due to the rapid development of deep convolutional neural networks (CNNs) [1, 2, 3].”
Knowledge Integration Networks for Action Recognition (2020, background): “Deep learning methods have achieved expressive performance on various vision tasks, such as human parsing (Gong et al. 2017), pose estimation (Wang et al. 2019), semantic segmentation (Zhao et al. 2017), and scene recognition (Zhou et al. 2017; Wang et al. 2017).”
CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images (2018, background): “Deep convolutional networks have rapidly advanced numerous computer vision tasks, providing state-of-the-art performance on image classification [9,31,34,14,37,8], object detection [28,27,22,20], sematic segmentation [23,11,4,10], etc.”
Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, background): “These metrics are calculated from a confusion matrix, which displays the crossing correct and wrong predictions between pairs of categories (classes) [68].”