Zero-Shot Detection via Vision and Language Knowledge Distillation

By Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin Cui. 2021

90 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models (2022, background): “The success has quickly inspired many follow-up large-scale pre-training works [68, 72, 69, 36, 43, 20, 34, 74].”
RegionCLIP: Region-based Language-Image Pretraining (2021, methodology): “On LVIS, following [18], we use the training/validation images for training/evaluation and adopt the category split with 866 base categories (common and frequent objects) and 337 novel categories (rare objects).”
Grounded Language-Image Pre-training (2021, methodology): “Our setting differs from zero-shot detection [1, 13, 43, 44, 63, 68], where some categories are defined as unseen/rare and not present in the training set.”
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model (2022, methodology): “Afterwards, we devise DetPro upon the recent OVOD pipeline ViLD [7] in Figure 2 where DetPro serves as a replacement for the proposal classifier in ViLD to realize automatic prompt engineering for it.”
GLIPv2: Unifying Localization and Vision-Language Understanding (2022, background): “A long line of research focuses on this generalization problem, under the name of zero-shot [4, 62, 7, 63], weakly-supervised [18, 5, 52], or open-vocabulary [58, 19] localization.”
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels (2021, methodology): “We utilize the pre-trained CLIP (ViT-B/32) text encoder to extract text embeddings of objects in our vocabulary and use the text prompts provided in [9] to ensemble the text representation.”
Detecting Twenty-thousand Classes using Image-level Supervision (2022, methodology): “Experiments on the open-vocabulary LVIS [15, 16] and the open-vocabulary COCO [2] benchmarks show that our method can significantly improve over a strong boxsupervised baseline, on both novel and base classes.”
Language-Grounded Indoor 3D Semantic Segmentation in the Wild (2022, methodology): “This allows for more flexibility in the feature learning by constraining only vector directions, and is similarly reflected in CLIP-driven image classification [27,16].”
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension (2022, methodology): “, 2021) and in object detection (Gu et al., 2021), but the same success has not yet been achieved in tasks requiring reasoning over vision and language.”
Decoupling Zero-Shot Semantic Segmentation (2021, methodology): “Following the previous works [19,46], for each category, we used multiple prompt templates to generate the text embeddings then ensemble these text embeddings by averaging.”
Open-Vocabulary Image Segmentation (2021, background): “Such a scenario has been recently developed and popularized for image classification [22, 38] and object detection [17, 50].”
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration (2022, methodology): “Zero-shot techniques have become increasingly popular in image classification [53, 33, 51], object detection [4, 54, 19, 38, 77, 42, 26], and semantic segmentation [6, 35, 32, 3, 14, 76].”
Image Segmentation Using Text and Image Prompts (2021, methodology): “Other approaches rely on CLIP for open-set object detection [46, 47].”
A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-Language Model (2021, unknown): “For example, Visual N-Grams [29] and CLIP [41] explored the use of web-crawled data for image classification and [19] introduced the vision-language pre-training model for the open-vocabulary object detection and showed it could significantly improve the long-tile object detection [22].”
FindIt: Generalized Localization with Natural Language Queries (2022, background): “Many recent approaches have started to study the open-set and zero-shot settings [15,2,84,98,20].”
VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, background): “, 2021), zero-shot object detection (Gu et al., 2021; Rahman et al., 2020) and incremental object detection (Liu et al.”
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic (2021, background): “With careful engineering of the prompt, one can, for example, improve detection of unseen objects [26].”