Detector discovery in the wild: Joint multiple instance and representation learning

By Judy Hoffman, Deepak Pathak, Trevor Darrell, Kate Saenko. 2014

78 citations recorded by the source corpus.

We develop methods for detector learning which exploit joint training over both weak (image-level) and strong (bounding box) labels and which transfer learned perceptual representations from strongly-labeled auxiliary tasks. Previous methods for weak-label learning often learn detector models independently using latent variable optimization, but fail to share deep representation knowledge across classes and usually require strong initialization. Other previous methods transfer deep representations from domains with strong labels to those with only weak labels, but do not optimize over individual latent boxes, and thus may miss specific salient structures for a particular category. We propose a model that subsumes these previous approaches, and simultaneously trains a representation and detectors for categories with either weak or strong labels present. We provide a novel formulation of a joint multiple instance learning method that includes examples from classification-style data when available, and also performs domain transfer learning to improve the underlying detector representation. Our model outperforms known methods on ImageNet-200 detection with weak labels.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully Convolutional Multi-Class Multiple Instance Learning (2014, methodology): “Hoffman et al. (2014) trains by MIL ﬁne-tuning but relies on bounding box proposals and supervised data for representation learning.”
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation (2016, methodology): “In detection, Hoffman et al . proposed a domain adaptation system by explicitly modeling the representation shift be-tween classiﬁcation and detection models [11] along with a follow-up work which incorporated per-category adaptation using multiple instance learning [12].”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “Hoffman et al. [60] develop a transfer learning-based algorithm, where the deep neural network is ﬁrst trained on both the weakly labeled auxiliary training data and the strongly labeled training data to obtain the background detector.”
Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation (2015, methodology): “In [35] and [14], deep learning features were incorporated into a MIL framework, and used to perform medical image analysis and object detection, respectively.”
Discover and Learn New Objects from Documentaries (2017, methodology): “The timeconsuming annotation process in object detection can be sidestepped by weakly supervised learning [2, 3, 7, 9, 13, 30, 33, 36, 40, 41, 42, 43, 44].”
Spatial Semantic Regularisation for Large Scale Object Detection (2015, unknown): “One major obstacle for large scale detection is the lack of bounding box annotations, which has been recently partially resolved by weakly supervised methods such as knowledge transfer [11], Multiple Instance Learning [27, 18], domain adaptation [12] or combined approaches [13].”
Augmenting Strong Supervision Using Web Data for Fine-Grained Categorization (2015, methodology): “Our method is also related to the strategy of training with auxiliary data sources, such as domain adaptation [16] for heterogeneous data sources and incremental learning [6, 30] for homogeneous ones.”
Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering (2016, methodology): “The MIL framework has been widely used in computer vision in problems where training data is often weakly supervised or incompletely labeled such as object detection [21,22], semantic segmentation [23,24], etc.”
Min-Entropy Latent Model for Weakly Supervised Object Detection (2018, methodology): “[21] trained detectors with weak annotations while transferring representations from extra object classes using full supervision (bounding-box annotation) and joint optimization.”
Multiple instance learning: A survey of problem characteristics and applications (2016, methodology): “In several methods for object localization, bags are composed of many candidate bounding boxes corresponding to instances [1, 59, 130, 131, 132].”
Cross-Domain Weakly-Supervised Object Detection Through Progressive Domain Adaptation (2018, methodology): “[13, 5] are the some of the best works closely related to this paper.”
DOCK: Detecting Objects by Transferring Common-Sense Knowledge (2018, methodology): “In particular, the recent state-of-the-art deep learning approaches of [17,18,39] adapt an image classifier into an object detector by learning a feature transformation between classifiers and detectors on the source classes, and transfer that transformation to related target classes based on visual”
Confident Learning: Estimating Uncertainty in Dataset Labels (2019, unknown): “uniformly random examples, on each of (a) the entire ILSVRC validation set, (b) the 20 (noisiest) classes with smallest diagonal in Cỹ,y∗ , (c) the foxhound class, which has the smallest diagonal in Cỹ,y∗ , and (d) the maillot class, a known erroneous class, duplicated accidentally in ImageNet, as p”