Detecting Twenty-thousand Classes using Image-level Supervision

By Xingyi Zhou, Rohit Girdhar, Armand Joulin, Phillip Krahenbuhl, Ishan Misra. 2022

887 citations recorded by the source corpus.

Current object detectors are limited in vocabulary size due to the small scale of detection datasets. Image classifiers, on the other hand, reason about much larger vocabularies, as their datasets are larger and easier to collect. We propose Detic, which simply trains the classifiers of a detector on image classification data and thus expands the vocabulary of detectors to tens of thousands of concepts. Unlike prior work, Detic does not need complex assignment schemes to assign image labels to boxes based on model predictions, making it much easier to implement and compatible with a range of detection architectures and backbones. Our results show that Detic yields excellent detectors even for classes without box annotations. It outperforms prior work on both open-vocabulary and long-tail detection benchmarks. Detic provides a gain of 2.4 mAP for all classes and 8.3 mAP for novel classes on the open-vocabulary LVIS benchmark. On the standard LVIS benchmark, Detic obtains 41.7 mAP when evaluated on all classes, or only rare classes, hence closing the gap in performance for object categories with few samples. For the first time, we train a detector with all the twenty-one-thousand classes of the ImageNet dataset and show that it generalizes to new datasets without finetuning. Code is available at \url{https://github.com/facebookresearch/Detic}.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploring Plain Vision Transformer Backbones for Object Detection (2022, methodology): “Detic [56] uses pre-trained CLIP [42] text embeddings.”
Simple Open-Vocabulary Object Detection with Vision Transformers (2022, methodology): “Many recent works aim to transfer the language capabilities of these models to object detection [12,26,45,46,20].”
ImageBind One Embedding Space to Bind Them All (2023, methodology): “Moreover, CLIP joint image and text embedding space has also been leveraged for a variety of zero-shot tasks like detection [23, 86], segmentation [40], mesh animation [79] etc.”
ReCo: Region-Controlled Text-to-Image Generation (2022, methodology): “We take the Detic object detector [49] to generate”
Visual-RFT: Visual Reinforcement Fine-Tuning (2025, neutral): “6, we compare Visual-RFT’s performance with several other detection models [41, 42, 50, 53, 59] us-ing nAP50 as the evaluation metric.”
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models (2023, background): “Most prior works on open-vocabulary segmentation either perform object detection with instance segmentation alone [17, 22, 23, 40, 51, 80, 81, 86, 89] or open-vocabulary semantic segmentation alone [22, 36, 76, 88].”
Scaling Open-Vocabulary Object Detection (2023, methodology): “2 – Human-curated vocabulary: 17 Detic [41] R50 – – – – 32.”
Vision-Language Models for Vision Tasks: A Survey (2023, background): “Open-vocabulary object detection [193] aims to detect objects described by arbitrary texts, i.e. , objects of any categories beyond the base classes.”
Self-regulating Prompts: Foundational Model Adaptation without Forgetting (2023, background): “VL models possess a strong understanding of open-vocabulary concepts, making them suitable for various downstream vision and vision-language applications [12, 56, 38, 30, 60, 13, 32, 53, 26, 36, 8].”
Transformer-Based Visual Segmentation: A Survey (2023, background): “Motivatedbythemorebalancedsam-plesfromimageclassiﬁcationdatasets,Detic[275]improvesthe performanceofthenovelclasseswithexistingimageclassiﬁca-tiondatasetsbysupervisingthemax-sizeproposalwithallimage labels.”
LERF: Language Embedded Radiance Fields (2023, methodology): “Detic [40] builds on existing two-stage object detector approaches, but demonstrates a greater generalization ability by allowing detector classiﬁers to train with image classiﬁ-cation data.”
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models (2022, methodology): “The leading DetPro (Du et al., 2022) shows the effectiveness of prompt optimization (Zhou et al., 2022b) which can potentially benefit F-VLM too.”
Open-Vocabulary DETR with Conditional Matching (2022, background): “Open-Vocabulary Object Detection leverages the recent advances in large pre-trained language models [35, 13, 37, 10] to incorporate the open-vocabulary information into object detectors.”
DETRs with Hybrid Matching (2022, methodology): “We follow the very recent work Detic [79] to only use the bounding box supervision for training.”