On learning to localize objects with minimal supervision

By Hyun Oh Song, Ross B. Girshick, Stefanie Sabrina Jegelka, Julien Mairal, Zaïd Harchaoui, Trevor Darrell. 2014

253 citations recorded by the source corpus.

Learning to localize objects with minimal supervision is an important problem in computer vision, since large fully annotated datasets are extremely costly to obtain. In this paper, we propose a new method that achieves this goal with only image-level labels of whether the objects are present or not. Our approach combines a discriminative submodular cover problem for automatically discovering a set of positive object windows with a smoothed latent SVM formulation. The latter allows us to leverage efficient quasi-Newton optimization techniques. Our experiments demonstrate that the proposed approach provides a 50% relative improvement in mean average precision over the current state-of-the-art on PASCAL VOC 2007 detection.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

From captions to visual concepts and back (2014, methodology): “Instead, we learn our detectors using the weakly-supervised approach of Multiple Instance Learning (MIL) [41, 46].”
LSDA: Large Scale Detection through Adaptation (2014, methodology): “A similar method [7] learns detectors from PASCAL VOC images without bounding box”
Contextual Action Recognition with R*CNN (2015, methodology): “[29] use MIL to localize objects with binary image-level labels (is the object present in the image or not).”
What's the Point: Semantic Segmentation with Point Supervision (2015, background): “The most common setting is where only image-level labels for the presence or absence of classes are provided during training [37, 38, 35, 26, 28, 40, 29] (Fig.”
Context as Supervisory Signal: Discovering Objects with Predictable Context (2014, methodology): “Instead, we use a greedy selection procedure reminiscent of [34] Specifically, given a selection χ of clusters, let sχi,j be the log likelihood ratio for the j’th highest-scoring patch in image i out of all clusters contained in χ.”
Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning (2015, neutral): “’14 [46] 27.”
Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals (2015, background): “Weakly-supervised localization [9, 12, 35, 36, 46, 49] shares the same type of output as colocalization, but assumes a more supervised scenario with image-level labels that indicate whether a target object class appears in the image or not.”
Region-Based Convolutional Networks for Accurate Object Detection and Segmentation (2016, methodology): “Song et al. [47] train an R-CNN using weak, image-level supervision by mining for positive training examples using a submodular cover algorithm and then training a latent SVM.”
Min-Entropy Latent Model for Weakly Supervised Object Detection (2018, methodology): “, an image was first decomposed into object proposals, with which clustering [14]–[16], latent variable learning [12]–[15], [17] or multiple instance learning [8], [10], [11], [21], [32] was used to perform proposal selection and classifier estimation.”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “Song-ICML-2014 [140] LSVM DeCAF None ImageNet(tag label) Graph model+LSVM* Initialization via discriminative submodular cover+smoothed LSVM learning General objects”
We Don’t Need No Bounding-Boxes: Training Object Class Detectors Using Only Human Verification (2016, methodology): “Recent work on WSOL [4, 5, 8, 48, 49, 60] has shown remarkable progress, mainly because of the use of Convolutional Neural Nets (CNN [20, 28]), which greatly improve visual recognition tasks.”
Attention-Based Dropout Layer for Weakly Supervised Object Localization (2019, background): “Existing approaches mine and track discriminative features of each class for object detection [45, 36, 37, 9, 45, 25, 21, 41, 19, 2, 39, 15, 63, 7, 5, 4, 48, 14, 65, 32, 31, 58, 62, 8, 6] and segmentation [33, 29, 18, 16, 24, 52, 50].”
Analysing Domain Shift Factors between Videos and Images for Object Detection (2015, unknown): “However, learning a detector with weak supervision is very challenging and current performance is still well below fully supervised methods [7], [38], [45].”
Track and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection (2016, methodology): “For this, we follow a similar approach to [31, 8, 33, 34].”