Self-taught object localization with deep networks

By Loris Bazzani, Alessandro Bergamo, Dragomir Anguelov, Lorenzo Torresani. 2014

160 citations recorded by the source corpus.

This paper introduces self-taught object localization, a novel approach that leverages deep convolutional networks trained for whole-image recognition to localize objects in images without additional human supervision, i.e., without using any ground-truth bounding boxes for training. The key idea is to analyze the change in the recognition scores when artificially masking out different regions of the image. The masking out of a region that includes the object typically causes a significant drop in recognition score. This idea is embedded into an agglomerative clustering technique that generates self-taught localization hypotheses. Our object localization scheme outperforms existing proposal methods in both precision and recall for small number of subwindow proposals (e.g., on ILSVRC-2012 it produces a relative gain of 23.4% over the state-of-the-art for top-1 hypothesis). Furthermore, our experiments show that the annotations automatically-generated by our method can be used to train object detectors yielding recognition results remarkably close to those obtained by training on manually-annotated bounding boxes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Webly Supervised Learning of Convolutional Networks (2015, methodology): “The outline of our approach is shown in Figure 2.”
Deep Learning Face Attributes in the Wild (2014, background): “Deep learning [18, 34, 23, 7, 19, 32, 31, 13, 33, 22, 3, 28] recently achieved great success in attribute prediction, due to their ability to learn compact and discriminative features.”
Object Detectors Emerge in Deep Scene CNNs (2014, background): “Another set of recent works (Oquab et al., 2014; Bergamo et al., 2014) demonstrate the ability of deep networks trained on object classiﬁcation to do localization without bounding box supervision.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, background): “This principle is applied for localization in [5].”
Learning Deep Features for Discriminative Localization (2015, background): “Recent work has shown that despite being trained on image-level labels, CNNs have the remarkable ability to localize objects [1, 16, 2, 15].”
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input (2018, background): “State of the art systems are trained using bounding box annotations for the training data (Girshick et al. 2013; Redmon et al. 2016), however other works investigate weakly-supervised or unsupervised object localization (Bergamo et al. 2014; Cho et al. 2015; Cinbis et al. 2016; Zhou et al. 2015).”
Visual Concept Recognition and Localization via Iterative Introspection (2016, background): “[12] Localizes image regions pertaining to the target class by masking out subimages and inspecting change in activations of the network.”
Risk Prediction with Electronic Health Records: A Deep Learning Approach (2016, background): “There are some works proposed to capture the temporal information in dynamic scenarios, such action recognition [29,30] and object localization [31] from video sequences.”
Weakly Supervised Cascaded Convolutional Networks (2016, background): “[18] introduced a class-specific object proposal generation based on the mask out strategy of [2], in order to have a reliable initialization.”
Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey (2018, methodology): “It is worth mentioning that one of the most recent studies [31] has proposed the class-specific OD approach, which has a similar objective as the COD approaches.”
Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization (2017, methodology): “This is in direct contrast to [1], which hides patches during testing but not during training.”
Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images (2017, background): “Another approach mentioned in [21] tries to identify the regions causing maximal activations while masking different portion of the images.”
Object Detection in 20 Years: A Survey (2019, unknown): “If the detection score drops sharply, then the masked region may contain an object with high probability [214].”
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition (2019, methodology): “Applying occlusion masks to input images (or features) and monitoring the changes at the output of a model has been used to visualizing whether a classifier can localize objects in an image [67], learning object detectors from weak image-level labels [3, 49, 36, 37], improving robustness of object d”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “Similar to [3], Bency et al. [4] propose a beam search algorithm to leverage the activation maps of a pre-trained classiﬁcation network to localize the objects of interest.”
Global Weighted Average Pooling Bridges Pixel-level Localization and Image-level Classification (2018, background): “Some works [26], [27] for this task heuristically search for regions that have important impacts on the classification.”
Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation (2016, background): “Nevertheless, as was noted in many recent research papers [21,49,31,3], deep image classification networks that were trained just from image-level labels, may be successfully employed to retrieve cues on object localization.”