DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

By Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun-Juan Zhu, Lionel Ming-shuan Ni, Harry Shum. 2022

2,919 citations recorded by the source corpus.

We present DINO (\textbf{D}ETR with \textbf{I}mproved de\textbf{N}oising anch\textbf{O}r boxes), a state-of-the-art end-to-end object detector. % in this paper. DINO improves over previous DETR-like models in performance and efficiency by using a contrastive way for denoising training, a mixed query selection method for anchor initialization, and a look forward twice scheme for box prediction. DINO achieves $49.4$AP in $12$ epochs and $51.3$AP in $24$ epochs on COCO with a ResNet-50 backbone and multi-scale features, yielding a significant improvement of $\textbf{+6.0}$\textbf{AP} and $\textbf{+2.7}$\textbf{AP}, respectively, compared to DN-DETR, the previous best DETR-like model. DINO scales well in both model size and data size. Without bells and whistles, after pre-training on the Objects365 dataset with a SwinL backbone, DINO obtains the best results on both COCO \texttt{val2017} ($\textbf{63.2}$\textbf{AP}) and \texttt{test-dev} (\textbf{$\textbf{63.3}$AP}). Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results. Our code will be available at \url{https://github.com/IDEACVR/DINO}.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

A Simple Framework for Open-Vocabulary Segmentation and Detection (2023, methodology): “Recently, Detection Transformer (DETR) [1] that is based on Transformer [44] has achieved significant progress in many detection [28, 34, 37, 55, 60] and segmentation models [5, 18, 29, 32].”
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, methodology): “Grounding DINO (Ours) DINO [58] A,B,C sub-sentence X zero-shot zero-shot zero-shot zero-shot”
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks (2022, neutral): “We compare with previous state-of-the-art models, including FD-SwinV2 [WHX22], DINO [ZLL22], Mask DINO [ZLL22], FD-CLIP [WHX22], CoCa [YWV22], OFA [WYM22], Florence [YCC21].”
MM-LLMs: Recent Advances in MultiModal Large Language Models (2024, neutral): “3 of Appendix G.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “The subﬁelds include point cloud segmentation, domain-aware segmentation, label and model efﬁcient segmentation, class agnostic segmentation, tracking, and medical segmentation.”
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions (2022, methodology): “For HTC++ and DINO, we train our models on COCO for 36 epochs (3× schedule) and does not introduce extra pre-training data like Object365 in [66].”
Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment (2022, methodology): “During testing, we adopt test time augmentation with various scales and their flipped counterparts and perform fusion 2 on the query features and the final predictions [35].”
Segment Anything in High Quality (2023, methodology): “For the COCO dataset, we use FocalNet-DINO [51] detector trained on COCO.”
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, background): “…advancements have been made across a variety of tasks, including image recognition [47, 31, 18, 83, 17], generic object detection [49, 87, 43, 27, 77, 36, 19, 38, 51, 50, 20, 30], generic image segmentation [9, 8, 26, 29, 78, 88, 79, 25, 79, 16, 28], referring object detection and segmentation…”
Recognize Anything: A Strong Image Tagging Model (2023, positive): “To address the issue of incorrect tags, we initially employ Grounding-DINO [28] to identify and crop regions corresponding to a specific category within all images.”
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation (2022, methodology): “Based on DAB-DETR and DN-DETR, DINO [32] proposes several new improvements on denoising and anchor refinement and achieves new SOTA results on COCO detection.”