Visual and Semantic Knowledge Transfer for Large Scale Semi-Supervised Object Detection

By Yuxing Tang, Josiah Wang, Xiaofang Wang, Boyang Gao, Emmanuel Dellandréa, Robert J. Gaizauskas, Liming Chen. 2016

115 citations recorded by the source corpus.

Deep CNN-based object detection systems have achieved remarkable success on several large-scale object detection benchmarks. However, training such detectors requires a large number of labeled bounding boxes, which are more difficult to obtain than image-level annotations. Previous work addresses this issue by transforming image-level classifiers into object detectors. This is done by modeling the differences between the two on categories with both image-level and bounding box annotations, and transferring this information to convert classifiers to detectors for categories without bounding box annotations. We improve this previous work by incorporating knowledge about object similarities from visual and semantic domains during the transfer process. The intuition behind our proposed method is that visually and semantically similar categories should exhibit more common transferable properties than dissimilar categories, e.g. a better detector would result by transforming the differences between a dog classifier and a dog detector onto the cat class, than would by transforming from the violin class. Experimental results on the challenging ILSVRC2013 detection dataset demonstrate that each of our proposed object similarity based knowledge transfer methods outperforms the baseline methods. We found strong evidence that visual similarity and semantic relatedness are complementary for the task, and when combined notably improve detection, achieving state-of-the-art detection performance in a semi-supervised setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Open-World Object Proposals without Learning to Classify (2021, methodology): “Approaches under semi-supervised setup [27,44,49,50,55,58,73] employ the weak image-level labels for novel classes as well as boxlevel labels for base classes.”
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation (2020, background): “udolabeled [42,71,34,3] video sequences. Semi-supervised learning has been widely applied to several computer vision tasks, including semantic segmentation [57,19,59,31,78,35,79,72], object detection [67,75,63], instance segmentation [35,60], panoptic segmentation [45], human pose estimation [58], p”
Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering (2019, background): “Another set of closely related work utilized additional data for scaling up either vision tasks (Hoffman et al., 2016; Tang et al., 2017; Redmon and Farhadi, 2017) or vision and language tasks (Venugopalan et al., 2017; Lu et al., 2018; Noh et al., 2019).”
Revisiting Knowledge Transfer for Training Object Class Detectors (2017, methodology): “On the opposite end of the spectrum, several works perform class-specific transfer [14, 31, 17, 42], For each target class, they determine a few most related source classes to transfer from.”
What leads to generalization of object proposals? (2020, background): “Multiple works [33,12,28,9] showed the benefit of sharing localization information between similar classes to improve semi supervised and weakly supervised detection.”
A Survey on Deep Semi-Supervised Learning (2021, background): “Recent works focusing on this direction have proposed [232], [233], [234], [235], [236] for DSSL.”
Consistency-based Semi-supervised Learning for Object detection (2019, methodology): “The weakly semi-supervised object detection method [8, 9] uses fully labeled data as well as weakly labeled data, as shown in Figure 1 (c).”