Dynamic Head: Unifying Object Detection Heads with Attentions

By Xiyang Dai, Yinpeng Chen, Bin Xiao, Dongdong Chen, Mengchen Liu, Lu Yuan, Lei Zhang. 2021

964 citations recorded by the source corpus.

The complex nature of combining localization and classification in object detection has resulted in the flourished development of methods. Previous works tried to improve the performance in various object detection heads but failed to present a unified view. In this paper, we present a novel dynamic head framework to unify object detection heads with attentions. By coherently combining multiple self-attention mechanisms between feature levels for scale-awareness, among spatial locations for spatial-awareness, and within output channels for task-awareness, the proposed approach significantly improves the representation ability of object detection heads without any computational overhead. Further experiments demonstrate that the effectiveness and efficiency of the proposed dynamic head on the COCO benchmark. With a standard ResNeXt-101-DCN backbone, we largely improve the performance over popular object detectors and achieve a new state-of-the-art at 54.0 AP. The code will be released at https://github.com/microsoft/DynamicHead.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Florence: A New Foundation Model for Computer Vision (2021, methodology): “We further compare Florence zero-shot with previous state-of-the-art detector4 (Dai et al., 2021a) (on COCO) fine-tunning on these tasks.”
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, background): “Grounding [62] FasterRCNN[39] - sentence partiallabel partiallabel FindIt[21] FasterRCNN[39] A sentence partiallabel ﬁne-tune MDETR[18] DETR[2] A,C word ﬁne-tune zero-shot ﬁne-tune DQ-DETR[46] DETR[2] A,C word zero-shot ﬁne-tune GLIP[26] DyHead[5] A word zero-shot zero-shot zero-shot GLIPv2[59] DyHe”
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models (2022, methodology): “• DyHead [8] represents a checkpoint for the traditional language-free object detector, where model is pre-trained on Object365 [53] without leveraging the category name information.”
Grounded Language-Image Pre-training (2021, methodology): “Though our current implementation is built upon Dynamic Head (DyHead) [10], our unified formulation can be generalized to any object detection systems [4, 6, 9, 10, 10, 36, 48, 49, 76].”
Exploring Plain Vision Transformer Backbones for Object Detection (2022, neutral): “Here we compare results that use ImageNet data (1K or 21K); better results are reported in [39,10] using extra data.”
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks (2022, neutral): “1 DyHead [DCX21] ImageNet-Pseudo Labels 2000 60.”
Focal Self-attention for Local-Global Interactions in Vision Transformers (2021, methodology): “We replace the Swin-Large model with our Focal-Large model, and use the same 2× training schedule as in [19].”
DaViT: Dual Attention Vision Transformers (2022, background): “Other operators in CNNs related to our work are Dynamic Head [12] and DANet [19].”
Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks (2023, methodology): “DyHead [12] generates pseudo labels on ImageNet and use the data as extra OD data.”
GLIPv2: Unifying Localization and Vision-Language Understanding (2022, methodology): “Following GLIP [36], we adopt Swin Transformer [40] as the image encoder Enc V , text transformers [51, 44] as the text encoder Enc L , Dynamic Head [15] with language-aware deep fusion [36] as the fusion encoder Enc V L , and Hourglass network [43] as instance segmentation head feature extractor.”
Rewrite the Stars (2024, neutral): “Our analysis could offer new insights into the effectiveness of self-attention and contribute to the revisiting on ‘dynamic’ features [7, 8, 12] in neural networks.”
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (2022, methodology): “…(ViTs) [35, 61, 89, 98], which have shown leading performances on many visual tasks – not only image classiﬁcation [35, 109] and representation learning [4, 10, 17, 105], but also many downstream tasks such as object detection [25,61], semantic segmentation [98, 103] and image restoration [11, 56].”
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone (2022, methodology): “With the same model architecture, by simply adding an object detection head ( e.g. , Dynamic Head [9]) on top, F IBER can be readily extended to visual grounding, referring expression comprehension and (open-vocabulary) OD tasks as well.”
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection (2022, methodology): “Distinct from GLIP [30], which introduces additional heavy modules like Dynamic Head [8] and cross-modal fusion, we directly adopt the vanilla ATSS [55] as our vision encoder to keep our architecture as neat as possible.”
Focal Attention for Long-Range Interactions in Vision Transformers (2021, methodology): “More recently, DyHead [17] achieves new SoTA on COCO, when combined with Swin-Large.”
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions (2022, neutral): “3 Florence-CoSwin-H# [59]DyHead [72] 637M 62.”