DETRs with Collaborative Hybrid Assignments Training

By Zhuofan Zong, Guanglu Song, Yu Liu. IEEE International Conference on Computer Vision, 2022

644 citations recorded by the source corpus.

In this paper, we provide the observation that too few queries assigned as positive samples in DETR with one-to-one set matching leads to sparse supervision on the encoder’s output which considerably hurt the discriminative feature learning of the encoder and vice visa for attention learning in the decoder. To alleviate this, we present a novel collaborative hybrid assignments training scheme, namely $\mathcal{C}o - {\text{DETR}}$, to learn more efficient and effective DETR-based detectors from versatile label assignment manners. This new training scheme can easily enhance the encoder’s learning ability in end-to-end detectors by training the multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS and Faster RCNN. In addition, we conduct extra customized positive queries by extracting the positive coordinates from these auxiliary heads to improve the training efficiency of positive samples in the decoder. In inference, these auxiliary heads are discarded and thus our method introduces no additional parameters and computational cost to the original detector while requiring no hand-crafted non-maximum suppression (NMS). We conduct extensive experiments to evaluate the effectiveness of the proposed approach on DETR variants, including DAB-DETR, Deformable-DETR, and DINO-Deformable-DETR. The state-of-the-art DINO-Deformable-DETR with Swin-L can be improved from 58.5% to 59.5% AP on COCO val. Surprisingly, incorporated with ViT-L backbone, we achieve 66.0% AP on COCO test-dev and 67.9% AP on LVIS val, outperforming previous methods by clear margins with much fewer model sizes. Codes are available at https://github.com/Sense-X/Co-DETR.

Subfields: Artificial Intelligence, Software

View Paper: https://www.semanticscholar.org/paper/3d3a76a2fedccea741f34303b06358440dc2a212

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

YOLOv10: Real-Time End-to-End Object Detection (2024, background): “Since then, various DETR variants have been proposed to enhance its performance and efficiency [42, 67, 56, 30, 36, 28, 5, 77, 82].”
Transformer-Based Visual Segmentation: A Survey (2023, background): “Meanwhile, there are also several works [182], [183], [184], [243] exploring query supervision from the target assignment perspective.”
GLaMM: Pixel Grounding Large Multimodal Model (2023, methodology): “For Level-1, we use Tag2Text [14] and RAM [58] for image tagging, Co-DETR [62], EVAv02 [7], OWL-ViT [33], and POMP [40] for object localization, GRiT [48] and GPT4RoI [57] for attribute generation, and MiDAS [39] for depth estimation.”
DETRs Beat YOLOs on Real-time Object Detection (2023, methodology): “Existing large DETR models [3, 6, 30, 38, 41, 43] have demonstrated impressive performance on COCO test-dev [19] leaderboard.”