Don’t Even Look Once: Synthesizing Features for Zero-Shot Detection

By Pengkai Zhu, Hanxiao Wang, Venkatesh Saligrama. 2019

101 citations recorded by the source corpus.

Zero-shot detection, namely, localizing both seen and unseen objects, increasingly gains importance for large-scale applications, with large number of object classes, since, collecting sufficient annotated data with ground truth bounding boxes is simply not scalable. While vanilla deep neural networks deliver high performance for objects available during training, unseen object detection degrades significantly. At a fundamental level, while vanilla detectors are capable of proposing bounding boxes, which include unseen objects, they are often incapable of assigning high-confidence to unseen objects, due to the inherent precision/recall tradeoffs that requires rejecting background objects. We propose a novel detection algorithm “Don’t Even Look Once (DELO),” that synthesizes visual features for unseen objects and augments existing training algorithms to incorporate unseen object detection. Our proposed scheme is evaluated on Pascal VOC and MSCOCO, and we demonstrate significant improvements in test accuracy over vanilla and other state-of-art zero-shot detectors

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

RegionCLIP: Region-based Language-Image Pretraining (2021, background): “Zeroshot object detection aims at detecting novel object classes that are not seen during detector training [2, 18, 38, 39, 60, 65].”
Grounded Language-Image Pre-training (2021, methodology): “Our setting differs from zero-shot detection [1, 13, 43, 44, 63, 68], where some categories are defined as unseen/rare and not present in the training set.”
Open-Vocabulary Object Detection Using Captions (2020, methodology): “…is to improve the Nevertheless, ZSD methods are still far from practical performance, due to their unnecessarily harsh constraint, i.e ., not having any example of unseen objects, and having to guess how they look like solely based on their word embed-dings [3, 30, 46] or textual descriptions [24].”
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels (2021, methodology): “We compare with recent zero-shot/open vocabulary methods [1,22,34,36].”
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models (2022, methodology): “Many methods address this by aligning the image region features to category word embeddings (Bansal et al., 2018; Rahman et al., 2020; Demirel et al., 2018; Zheng et al., 2020), or synthesizing visual features with a generative model (Hayat et al., 2020; Zhu et al., 2020).”
GLIPv2: Unifying Localization and Vision-Language Understanding (2022, background): “A long line of research focuses on this generalization problem, under the name of zero-shot [4, 62, 7, 63], weakly-supervised [18, 5, 52], or open-vocabulary [58, 19] localization.”
Open-Vocabulary DETR with Conditional Matching (2022, methodology): “For completeness, we also list the results of some previous zero-shot methods such as SB [1], DELO [38] and PL [27] in Table 4 rows 1-3.”