DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

By Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang. 2022

1,335 citations recorded by the source corpus.

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This new formulation directly uses box coordinates as queries in Transformer decoders and dynamically updates them layer-by-layer. Using box coordinates not only helps using explicit positional priors to improve the query-to-feature similarity and eliminate the slow training convergence issue in DETR, but also allows us to modulate the positional attention map using the box width and height information. Such a design makes it clear that queries in DETR can be implemented as performing soft ROI pooling layer-by-layer in a cascade manner. As a result, it leads to the best performance on MS-COCO benchmark among the DETR-like detection models under the same setting, e.g., AP 45.7\% using ResNet50-DC5 as backbone trained in 50 epochs. We also conducted extensive experiments to confirm our analysis and verify the effectiveness of our methods. Code is available at \url{https://github.com/SlongLiu/DAB-DETR}.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Simple Framework for Open-Vocabulary Segmentation and Detection (2023, methodology): “Recently, Detection Transformer (DETR) [1] that is based on Transformer [43] has achieved significant progress in many detection [27, 33, 36, 54, 59] and segmentation models [5, 18, 28, 31].”
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, methodology): “Motivated by the encouraging progress of Transformerbased detectors [24, 25, 31, 58], in this work, we propose to build a strong open-set detector based on DINO [58], which not only offers the state-of-the-art object detection performance, but also allows us to integrate multi-level text information”
OneFormer: One Transformer to Rule Universal Image Segmentation (2022, methodology): “Architectures based on the transformer encoder-decoder structure [2,25,28,51] have proved effective in object detection since the introduction of DETR [2].”
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, background): “…have been made across a variety of tasks, including image recognition [47, 31, 18, 83, 17], generic object detection [49, 87, 43, 27, 77, 36, 19, 38, 51, 50, 20, 30], generic image segmentation [9, 8, 26, 29, 78, 88, 79, 25, 79, 16, 28], referring object detection and segmentation [41,…”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, background): “[50,30,26] generate the object queries from anchor points or anchors that use position prior for fast convergence.”
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding (2022, methodology): “Many follow-ups (Zhu et al. 2021; Meng et al. 2021; Gao et al. 2021; Dai et al. 2021; Wang et al. 2021; Liu et al. 2022; Li et al. 2022; Zhang et al. 2022) improve the model performance and training efﬁciency of DETR.”
DETRs with Hybrid Matching (2022, methodology): “Motivated by the success of DETR on a wide variety of vision tasks, many follow-up efforts have improved DETR from various aspects, including redesigning more advanced transformer encoder [81, 13, 14] or transformer decoder architectures [44, 73, 81, 4] or query formulations [62, 34, 22, 74].”
Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition (2021, background): “Very recently, seminal works based on self-attention mechanisms have emerged, such as Swin-Transformer [26], DAB-DETR [27], DaViT [28], and QFormer [29].”
Transformer-Based Visual Segmentation: A Survey (2023, background): “DAB-DETR [177] ﬁnds the localization issues of the learnable query and proposes dynamic anchor boxes to replace the learnable query.”
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022, methodology): “1 − − − − 196 40M 24 DAB-DETR(DC5)† [21] 12 38.”
Attention mechanisms in computer vision: A survey (2021, neutral): “[8], [9], [15], [21], [31], [32], [34], [35] , [22], [26], [62], [63], [64], [65], [66], [67] , [41], [68], [69], [70], [71], [72], [73], [74] , [8], [34], [42], [43], [75], [76], [77], [78] , [27], [44], [45], [46], [79], [80], [81], [82] , [61], [83], [84], [85], [86], [87], [88], [89] , [47], [90”
YOLOv10: Real-Time End-to-End Object Detection (2024, background): “Since then, various DETR variants have been proposed to enhance its performance and efficiency [42, 67, 56, 30, 36, 28, 5, 77, 82].”
DiffusionDet: Diffusion Model for Object Detection (2022, background): “Since then, the query-based detection paradigm has attracted great attention and inspired a series of following works [12,24,43,51,58,64,66,89,90,107,110, 114].”
DETRs with Collaborative Hybrid Assignments Training (2022, methodology): “DINO [39] inher-its the advanced query formulation of DAB-DETR [23] and incorporates an improved contrastive denoising technique to achieve state-of-the-art performance.”
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation (2022, methodology): “First, its queries follow the design in DETR [1] without being able to utilize better positional priors as studied in Conditional DETR [26], Anchor DETR [34], and DABDETR [22].”
DETRs Beat YOLOs on Real-time Object Detection (2023, background): “Recently, the end-to-end Transformer-based detectors (DETRs) [4, 16, 22, 25, 33, 36, 41, 42] have received extensive attention from the academia due to their streamlined architecture and elimination of hand-crafted components.”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, methodology): “Our approach is inspired by DETR [4] and its follow-up works [63, 32, 56, 25, 28, 10, 59], especially DAB-DETR [28], where the object query is considered as the positional embedding of a spatial anchor box.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “DETR3D [39] and PETR [24] conduct the 3D object detection mainly inspired by the end-to-end DETR methods [4,23,28,46].”
Visual attention network (2022, background): “, vision transformers) [13], [15], [20], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64] have achieved significantly better performance than the mainstream CNNs on different visual tasks, showing the huge potential of attention-based models.”
YOLOv12: Attention-Centric Real-Time Object Detectors (2025, neutral): “Recently, an end-to-end object detection method, namely RT-DETR [66], improved traditional end-to-end detectors [7, 33, 37, 42, 71] to meet real-time requirements by designing an efficient encoder and an uncertainty-minimal query selection mechanism.”