Weakly Supervised Object Localization Using Things and Stuff Transfer

By Miaojing Shi, Holger Caesar, Vittorio Ferrari. 2017

75 citations recorded by the source corpus.

We propose to help weakly supervised object localization for classes where location annotations are not available, by transferring things and stuff knowledge from a source set with available annotations. The source and target classes might share similar appearance (e.g. bear fur is similar to cat fur) or appear against similar background (e.g. horse and sheep appear against grass). To exploit this, we acquire three types of knowledge from the source set: a segmentation model trained on both thing and stuff classes; similarity relations between target and source classes; and cooccurrence relations between thing and stuff classes in the source. The segmentation model is used to generate thing and stuff segmentation maps on a target image, while the class similarity and co-occurrence knowledge help refining them. We then incorporate these maps as new cues into a multiple instance learning framework (MIL), propagating the transferred knowledge from the pixel level to the object proposal level. In extensive experiments, we conduct our transfer from the PASCAL Context dataset (source) to the ILSVRC, COCO and PASCAL VOC 2007 datasets (targets). We evaluate our transfer across widely different thing classes, including some that are not similar in appearance, but appear against similar background. The results demonstrate significant improvement over standard MIL, and we outperform the state-of-the-art in the transfer setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

COCO-Stuff: Thing and Stuff Classes in Context (2016, background): “Moreover, several works have shown that stuff is a useful contextual cue to detect things and vice versa [41, 27, 31, 38, 45].”
Consistency-based Semi-supervised Learning for Object detection (2019, methodology): “Weakly supervised object detection methods [4, 5, 6, 7] are required to learn only with image-level labeled data, as shown in Figure 1 (b).”
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection (2018, methodology): “Many previous methods follow the Multiple Instance Learning (MIL) pipeline for WSOD [12], [13], [14], [15], [16], [17], [18], [19].”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “The methods of this category ﬁne-tune the off-the-shelf DNN models during the weakly supervised learning process to obtain strong object detectors [16], [123], [179].”
Attention-Based Dropout Layer for Weakly Supervised Object Localization (2019, background): “Existing approaches mine and track discriminative features of each class for object detection [45, 36, 37, 9, 45, 25, 21, 41, 19, 2, 39, 15, 63, 7, 5, 4, 48, 14, 65, 32, 31, 58, 62, 8, 6] and segmentation [33, 29, 18, 16, 24, 52, 50].”