Object Detectors Emerge in Deep Scene CNNs

By Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, Antonio Torralba. 2014

1,327 citations recorded by the source corpus.

With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. One important factor for continued progress is to understand the representations that are learned by the inner layers of these deep architectures. Here we show that object detectors emerge from training CNNs to perform scene classification. As scenes are composed of objects, the CNN for scene classification automatically discovers meaningful objects detectors, representative of the learned scene categories. With object detectors emerging as a result of learning to recognize scenes, our work demonstrates that the same network can perform both scene recognition and object localization in a single forward-pass, without ever having been explicitly taught the notion of objects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

What makes ImageNet good for transfer learning? (2016, methodology): “Understanding internal representations of CNN: For understanding what information is encoded by different CNN features, several works have developed feature visualization methods using either deconvolution [50], backpropagation[40, 52] or by training a decoder network [14].”
Making Convolutional Networks Shift-Invariant Again (2019, background): “Analysis of deep networks have included qualitative approaches, such as showing patches which activate hidden units (Girshick et al., 2014; Zhou et al., 2015), actively maximizing hidden units (Mordvintsev et al., 2015), and mapping features back into pixel space (Zeiler & Fergus, 2014; H´enaff &…”
Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces (2017, positive): “We further analyze the qualities of the learned features by visualizing the receptive ﬁeld segmentation of TextTopicNet convolutional units using the methodology of [45, 25].”
Unified Perceptual Parsing for Scene Understanding (2018, methodology): “To overcome the issue raised by Zhou et al. [32] that although the theoretical receptive ﬁeld of deep CNN is large enough, the empirical receptive ﬁeld of deep CNN is relatively much smaller [33], we apply a Pyramid Pooling Module (PPM) from PSPNet [16] on the last layer of the backbone network…”
SSD: Single Shot MultiBox Detector (2015, background): “Feature maps from different levels within a network are known to have different (empirical) receptive field sizes [24].”
Learning to decompose for object detection and instance segmentation (2015, background): “C V ] 1 1 M ay 2 01 6 (2014); Zhou et al. (2015) and others have shown the ability to localize objects in images.”
Generating Videos with Scene Dynamics (2016, methodology): “Figure 6: Visualizing Representation: We visualize some hidden units in the encoder of the future generator, following the technique from [52].”
Deep Learning Face Attributes in the Wild (2014, background): “Deep learning [18, 34, 23, 7, 19, 32, 31, 13, 33, 22, 3, 28] recently achieved great success in attribute prediction, due to their ability to learn compact and discriminative features.”
Pyramid Scene Parsing Network (2016, background): “We contrarily propose generating initial results by supervision with an additional loss, and learning the residue afterwards with the final loss.”
Learning to Generate Reviews and Discovering Sentiment (2017, background): “Analysis of the task speciﬁc representations learned by these models reveals many fascinating properties (Zhou et al., 2014).”
ParseNet: Looking Wider to See Better (2015, background): “As shown in [32], the actual sizes are about 25% of the theoricatical ones at higher levels, thus preventing the model from making global decisions.”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, background): “This can be perhaps be attributed to the smaller empirical size of the receptive field of the feature units in the deepest encoder layer size [58].”
Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning (2017, positive): “We assumed a circular receptive ﬁeld, obtaining its radius from Zhou et al (2015).”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, background): “There have been hypotheses presented in the literature [60, 57] that neurons act as concept ‘detectors’.”
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks (2016, methodology): “Interestingly, our visualizations produce similar results to the method in [28] that requires showing each neuron a large, external dataset of images to discover what feature each neuron has learned to detect (Fig.”
Learning Deep Features for Discriminative Localization (2015, methodology): “Recent work by Zhou et al [33] has shown that the convolutional units of various layers of convolutional neural networks (CNNs) actually behave as object detectors despite no supervision on the location of the object was provided.”
Understanding Neural Networks Through Deep Visualization (2015, background): “These visualizations suggest that further study into the exact nature of learned representations — whether they are local to a single channel or distributed across several — is likely to be interesting (see Zhou et al. (2014) for work in this direction).”
Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network (2017, background): “However, studies [39] show that network tends to gather information mainly from a much smaller region in the receptive field, which is called valid receptive field (VRF) in this paper.”