Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning

By R. G. Cinbis, Jakob Verbeek, Cordelia Schmid. 2015

441 citations recorded by the source corpus.

Object category localization is a challenging problem in computer vision. Standard supervised training requires bounding box annotations of object instances. This time-consuming annotation process is sidestepped in weakly supervised learning. In this case, the supervised information is restricted to binary labels that indicate the absence/presence of object instances in the image, without their locations. We follow a multiple-instance learning approach that iteratively trains the detector and infers the object locations in the positive training images. Our main contribution is a multi-fold multiple instance learning procedure, which prevents training from prematurely locking onto erroneous object locations. This procedure is particularly important when using high-dimensional representations, such as Fisher vectors and convolutional neural network features. We also propose a window refinement method, which improves the localization accuracy by incorporating an objectness prior. We present a detailed experimental evaluation using the PASCAL VOC 2007 dataset, which verifies the effectiveness of our approach.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Deep Features for Discriminative Localization (2015, background): “Recent work has shown that despite being trained on image-level labels, CNNs have the remarkable ability to localize objects [1, 16, 2, 15].”
Open-Vocabulary Object Detection Using Captions (2020, background): “This problem is typically addressed using multiple instance learning, which is a well-studied topic [4, 39, 8].”
Attend Refine Repeat: Active Box Proposal Generation via In-Out Localization (2016, methodology): “Object proposals have also been used in various other tasks, such as weakly-supervised object detection [7], exemplar 2D-3D detection [30], visual semantic role labelling [17], caption generation [24] or visual question answering [37].”
Towards Weakly-Supervised Action Localization (2016, methodology): “One issue is that it tends to lock onto the initial locations [17], which is particularly the case for high dimensional descriptors.”
Learning Word-Like Units from Joint Audio-Visual Analysis (2017, background): “While supervised object detection is a standard problem in the vision community, several recent works have tackled the problem of weakly-supervised or unsupervised object localization (Bergamo et al., 2014; Cho et al., 2015; Zhou et al., 2015; Cinbis et al., 2016).”
Min-Entropy Latent Model for Weakly Supervised Object Detection (2018, methodology): “, an image was first decomposed into object proposals, with which clustering [14]–[16], latent variable learning [12]–[15], [17] or multiple instance learning [8], [10], [11], [21], [32] was used to perform proposal selection and classifier estimation.”
Global Weighted Average Pooling Bridges Pixel-level Localization and Image-level Classification (2018, methodology): “Weakly supervised object detection methods [15], [16], [17], [7], [8], [9], [10] are evolved from the region-based object detection frameworks [18], [19], [20], [21].”
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection (2018, methodology): “Many methods are emerging for the WSOD problem [13], [14], [33], [34], [35], [36], [37], [38], [39], [40], [58], [59], [60], [61].”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, background): “Some researchers [47], [48], [49], [50], [51], [52] attempt to solve WSOL as a multiple instance learning framework.”
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning (2020, background): “In addition, deep MIL models have been considered for other weakly supervised vision tasks, including weakly supervised object localization [9] and detection [45, 49].”
From Discriminant to Complete: Reinforcement Searching-Agent Learning for Weakly Supervised Object Detection (2020, background): “Some of these approaches [4]–[6], [8], [9], [28], [29] adopt the deep features extracted from the pretrained deep neural networks to facilitate the learning process.”
Multiple instance learning: A survey of problem characteristics and applications (2016, background): “Also, several authors noted that in this context, MIL algorithms are sensitive to initialization [9,107].”