Simple Multi-dataset Detection

By Xingyi Zhou, Vladlen Koltun, Philipp Krahenbuhl. 2021

160 citations recorded by the source corpus.

How do we build a general and broad object detection system? We use all labels of all concepts ever annotated. These labels span diverse datasets with potentially inconsistent taxonomies. In this paper, we present a simple method for training a unified detector on multiple large-scale datasets. We use dataset-specific training protocols and losses, but share a common detection architecture with dataset-specific outputs. We show how to automatically integrate these dataset-specific outputs into a common semantic taxonomy. In contrast to prior work, our approach does not require manual taxonomy reconciliation. Experiments show our learned taxonomy outperforms a expert-designed taxonomy in all datasets. Our multi-dataset detector performs as well as dataset-specific models on each training domain, and can generalize to new unseen dataset without fine-tuning on them. Code is available at https://github.com/xingyizhou/UniDet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Florence: A New Foundation Model for Computer Vision (2021, methodology): “For Object365, we use the same input resolution of images (i.e. , the maximum image side 1, 333) as the Multi-dataset Detec- tion1 (Zhou et al., 2021) for fine-tuning.”
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation (2021, background): “Zhao et al. (2020) and Zhou et al. (2021) unify the label space from multiple datasets.”
Detecting Twenty-thousand Classes using Image-level Supervision (2022, methodology): “We group images from the same dataset on the same GPU to improve training efficiency [77].”
LMSeg: Language-guided Multi-dataset Segmentation (2023, methodology): “Inspired by CoOp (Zhou et al., 2022b), when generating the text embedings k text , we utilize a learnable prompt, namely “ , where L is the length of the learnable vectors and is set to 8 by default.”
T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation (2023, unknown): “We first use UniDet [46] to detect objects in the generated image.”
Transformer-Based Visual Segmentation: A Survey (2023, background): “Recently, to enhance instance-level context, Wang et al. [342] adds an extra query from Mask2Former into memory matching for VOS.”
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension (2022, methodology): “ReCLIP w/o relations is 1-2% better than ReCLIP in the settings with ground-truth proposals and ﬁltered UniDet proposals.”
DAMO-YOLO : A Report on Real-Time Object Detection Design (2022, background): “This reduces the original 945 categories (80 from COCO, 365 from Objects365, and 500 from OpenImage) to 701 [42].”