DETRs with Hybrid Matching

By Ding Jia, Yuhui Yuan, Hao He, Xiao-pei Wu, Haojun Yu, Weihong Lin, Lei-huan Sun, Chao Zhang, Hanhua Hu. 2022

325 citations recorded by the source corpus.

One-to-one set matching is a key design for DETR to establish its end-to-end capability, so that object detection does not require a hand-crafted NMS (non-maximum suppression) to remove duplicate detections. This end-to-end signature is important for the versatility of DETR, and it has been generalized to broader vision tasks. However, we note that there are few queries assigned as positive samples and the one-to-one set matching significantly reduces the training efficacy of positive samples. We propose a simple yet effective method based on a hybrid matching scheme that combines the original one-to-one matching branch with an auxiliary one-to-many matching branch during training. Our hybrid strategy has been shown to significantly improve accuracy. In inference, only the original one-to-one match branch is used, thus maintaining the end-to-end merit and the same inference efficiency of DETR. The method is namedℋ-DETR, and it shows that a wide range of representative DETR methods can be consistently improved across a wide range of visual tasks, including Deformable-DETR, PETRv2, PETR, and TransTrack, among others. Code is available at: https://github.com/HDETR.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lei Zhang
2. Ziwei Liu
3. Chen Change Loy
4. Jianwei Yang
5. Ping Luo
6. Feng Li
7. Hao Zhang
8. Kai Chen

In Computer Vision and Pattern Recognition

1. Lei Zhang
2. Dacheng Tao
3. Chen Change Loy
4. Luc Van Gool
5. Xiaogang Wang
6. Ping Luo
7. Ming-Hsuan Yang
8. Jianwei Yang

In Model Architectures

1. Jianwei Yang
2. Feng Li
3. Hao Zhang
4. Lei Zhang
5. Shilong Liu
6. Guiguang Ding
7. Yu Liu
8. Guanglu Song

Citations in the corpus

• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, background): “DETR was first proposed in [2] and then has been improved from many directions [4, 5, 12, 17, 33, 50, 64] in the past few years.”
• DiffusionDet: Diffusion Model for Object Detection (2022, background): “Since then, the query-based detection paradigm has attracted great attention and inspired a series of following works [12,24,43,51,58,64,66,89,90,107,110, 114].”
• Transformer-Based Visual Segmentation: A Survey (2023, methodology): “It proposes to distill unsupervised features into high-quality discrete semantic labels.”
• DETRs with Collaborative Hybrid Assignments Training (2022, methodology): “Moreover, we conduct experiments on the improved Deformable-DETR (denoted as Deformable-DETR++) following [16], where a +2.”
• Rep ViT: Revisiting Mobile CNN From ViT Perspective (2023, methodology): “To ensure the inference efficiency on the mobile device, we prioritize the simple 3 × 3 convolutions in all modules.”
• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, background): “…have been made across a variety of tasks, including image recognition [47, 31, 18, 83, 17], generic object detection [49, 87, 43, 27, 77, 36, 19, 38, 51, 50, 20, 30], generic image segmentation [9, 8, 26, 29, 78, 88, 79, 25, 79, 16, 28], referring object detection and segmentation [41, 37,…”
• YOLOv10: Real-Time End-to-End Object Detection (2024, background): “Since then, various DETR variants have been proposed to enhance its performance and efficiency [42, 67, 56, 30, 36, 28, 5, 77, 82].”