Zero Shot Detection

By Pengkai Zhu, Hanxiao Wang, Venkatesh Saligrama. 2018

108 citations recorded by the source corpus.

As we move toward large-scale object detection, it is unrealistic to expect annotated training data, in the form of bounding box annotations around objects, for all object classes at sufficient scale; therefore, the methods capable of unseen object detection are required. We propose a novel zero-shot method based on training an end-to-end model that fuses semantic attribute prediction with visual features to propose object bounding boxes for seen and unseen classes. While we utilize semantic features during training, our method is agnostic to semantic information for unseen classes at test-time. Our method retains the efficiency and effectiveness of YOLOv2 for objects seen during training, while improving its performance for novel and unseen objects. The ability of the state-of-the-art detection methods to learn discriminative object features to reject background proposals also limits their performance for unseen objects. We posit that, to detect unseen objects, we must incorporate semantic information into the visual domain so that the learned visual features reflect this information and lead to improved recall rates for unseen objects. We test our method on PASCAL VOC and MS COCO dataset and observed significant improvements on the average precision of unseen classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Grounded Language-Image Pre-training (2021, background): “…of work identiﬁes building a open-world object proposal module that could propose any novel object at test time as the key challenge [22, 25, 60, 74, 75]; GLIP offers a new perspective: the model does not need to propose every possible novel objects from an open set; rather it only needs to…”
Incremental Few-Shot Object Detection (2020, neutral): “There are other techniques that aim to minimise the amount of data labelling for object detection such as weakly supervised learning [4, 10] and zero-shot learning [58, 37, 2].”
Open-Vocabulary Object Detection Using Captions (2020, background): “[45, 46] argued the key to ZSD is to improve the Nevertheless, ZSD methods are still far from practical performance, due to their unnecessarily harsh constraint, i.e ., not having any example of unseen objects, and having to guess how they look like solely based on their word embed-dings [3, 30,…”
Few-Shot Object Detection via Feature Reweighting (2018, background): “Zeroshot object detection [1, 27, 42] aims to detect previously unseen object categories, thus usually requires external information such as relations between classes.”
Zero-Shot Object Detection (2018, neutral): “Comparison with recent works on ZSD : After completion of this work, we found two parallel works by Zhu et al. [61] and Rahman et al. [42] that target a similar problem.”
Zero-Shot Semantic Segmentation (2019, background): “e synthetic instances of unseen classes by training a conditional generator from the seen classes. There are very few extensions of zero-shot learning to other tasks than classiﬁcation. Very recently [4, 14, 35, 51] attacked object detection: unseen objects are detected while box annotation is only ”
FindIt: Generalized Localization with Natural Language Queries (2022, background): “Many recent approaches have started to study the open-set and zero-shot settings [15,2,84,98,20].”
Small Sample Learning in Big Data Era (2018, background): “In the future, more other problems are worthy to be investigated, such as generation (Lake et al., 2015), synthesis (Long et al., 2017c), parsing (Zhu et al., 2018a; Bansal et al., 2018; Rahman et al., 2018; Shaban et al., 2017), account for concepts are far more complicated than object categories…”
Probing Contextual Language Models for Common Ground with Visual Representations (2020, background): “, 2018), visual attributes (Demirel et al., 2018; Zhu et al., 2019; Mao et al., 2020) or text descriptions (Li et al.”
Any-Shot Object Detection (2020, methodology): “For example, [3], [9, 43] and [27] employ pre-computed object proposals [44], YOLOv2 [29] and Faster-RCNN [30] based methods for ZSD, respectively.”
Transductive Learning for Zero-Shot Object Detection (2019, background): “Existing efforts [1, 3, 24, 38, 22] to address the ZSD problem explore”
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model (2022, background): “Despite they have made some progress, their overall performance is still far behind the fully-supervised methods [1, 39], therefore research on it is not flourishing yet.”
Meta-Learning to Detect Rare Objects (2019, background): “There are other settings of detection with limited supervision, such as weakly-supervised detection [59, 5, 11], semi-supervised detection [69, 39, 12], and zero-shot detection [2, 41, 73], which are different from ours.”
Probing Text Models for Common Ground with Visual Representations (2020, background): “, 2018), visual attributes (Demirel et al., 2018; Zhu et al., 2019; Mao et al., 2020) or text descriptions (Li et al.”
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels (2021, methodology): “Recent methods encourage the visual-semantic alignment for novel objects by different strategies such as synthesizing visual representations of novel classes [35, 36] or uti-”
Improved Visual-Semantic Alignment for Zero-Shot Object Detection (2020, methodology): “The ZSD problem has recently been investigated by (Rahman, Khan, and Porikli 2018b; Zhu et al. 2018; Bansal et al. 2018; Demirel, Cinbis, and Ikizler-Cinbis 2018; Li et al. 2019).”
GLIPv2: Unifying Localization and Vision-Language Understanding (2022, background): “A long line of research focuses on this generalization problem, under the name of zero-shot [4, 62, 7, 63], weakly-supervised [18, 5, 52], or open-vocabulary [58, 19] localization.”