Scene Recognition with CNNs: Objects, Scales and Dataset Bias

By Luis Herranz, Shuqiang Jiang, Xiangyang Li. 2016

225 citations recorded by the source corpus.

Since scenes are composed in part of objects, accurate recognition of scenes requires knowledge about both scenes and objects. In this paper we address two related problems: 1) scale induced dataset bias in multi-scale convolutional neural network (CNN) architectures, and 2) how to combine effectively scene-centric and object-centric knowledge (i.e. Places and ImageNet) in CNNs. An earlier attempt, Hybrid-CNN[23], showed that incorporating ImageNet did not help much. Here we propose an alternative method taking the scale into account, resulting in significant recognition gains. By analyzing the response of ImageNet-CNNs and Places-CNNs at different scales we find that both operate in different scale ranges, so using the same network for all the scales induces dataset bias resulting in limited performance. Thus, adapting the feature extractor to each particular scale (i.e. scale-specific CNNs) is crucial to improve recognition, since the objects in the scenes have their specific range of scales. Experimental results show that the recognition accuracy highly depends on the scale, and that simple yet carefully chosen multi-scale combinations of ImageNet-CNNs and Places-CNNs, can push the stateof-the-art recognition accuracy in SUN397 up to 66.26% (and even 70.17% with deeper architectures, comparable to human performance).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Do Better ImageNet Models Transfer Better? (2018, methodology): “works pretrained on the Places dataset [29, 77], which more closely matches the domain of SUN397.”
Deep learning for visual understanding (2017, unknown): “Compared with another recent work [255], our method achieves better”
Deep convolutional neural networks for mobile capture device-based crop disease classification in the wild (2019, methodology): “Agricultural applications are not an exception to this trend, and deep neural nets have been successfully been used to predict land use from remote sensing images (Hu et al. (2015)), plant phenotyping (Pound et al.”
Planning and Decision-Making for Autonomous Vehicles (2018, background): “Nonetheless, both approaches result in increased data set bias, which can also be found in real-world data sets (64).”
Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks? (2016, background): “…systems, where combinations of codified rules and learned classifiers are applied to each of the individual sub-tasks involved in a complex action like driving, to end-to-end learning, in which sensor data is mapped directly to the action space of driving through learning from many examples.”
Learning More Universal Representations for Transfer-Learning (2017, methodology): “For instance, variations on the image set while preserving the set of categories have been proposed in [21], [43]; adding variations has been used by [4], [37], [38] (by adding data labeled among speciﬁc categories) and [31], [49] (by adding data labeled among generic categories); splitting…”