RTMDet: An Empirical Study of Designing Real-Time Object Detectors

By Chengqi Lyu, Wenwei Zhang, Haian Huang, Yue Zhou, Yudong Wang, Yanyi Liu, Shilong Zhang, Kai Chen. arXiv.org, 2022

862 citations recorded by the source corpus.

In this paper, we aim to design an efficient real-time object detector that exceeds the YOLO series and is easily extensible for many object recognition tasks such as instance segmentation and rotated object detection. To obtain a more efficient model architecture, we explore an architecture that has compatible capacities in the backbone and neck, constructed by a basic building block that consists of large-kernel depth-wise convolutions. We further introduce soft labels when calculating matching costs in the dynamic label assignment to improve accuracy. Together with better training techniques, the resulting object detector, named RTMDet, achieves 52.8% AP on COCO with 300+ FPS on an NVIDIA 3090 GPU, outperforming the current mainstream industrial detectors. RTMDet achieves the best parameter-accuracy trade-off with tiny/small/medium/large/extra-large model sizes for various application scenarios, and obtains new state-of-the-art performance on real-time instance segmentation and rotated object detection. We hope the experimental results can provide new insights into designing versatile real-time object detectors for many object recognition tasks. Code and models are released at https://github.com/open-mmlab/mmdetection/tree/3.x/configs/rtmdet.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/99ccc15094b9438448668d136d90a12e0c9443e1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Zijia Lin
2. Qibin Hou
3. Guiguang Ding
4. Jungong Han
5. Hui Chen
6. Ao Wang
7. Xiang-Yang Li
8. Jian Yang

Citations in the corpus

• Hyper-YOLO: When Visual Object Detection Meets Hypergraph Computation (2024, neutral): “DETR [25], [27], [28], [29], [30], built on the Transformer architecture, has emerged as a powerful alternative, surpassing YOLO in terms of both speed and accuracy when employing large-scale models.”
• Large Selective Kernel Network for Remote Sensing Object Detection (2023, methodology): “Our LSKNet achieves the best mAP under similar complexity budgets, whilst surpassing the previous best public records [43].”
• RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose (2023, methodology): “By replacing the ResNet-50 [21] backbone with the more compact CSPNext-m [42], we further reduce the model size and obtain a lightweight yet strong baseline, 69.”
• YOLOv10: Real-Time End-to-End Object Detection (2024, background): “Over the past years, substantial efforts have been directed towards developing efficient detectors [19, 57, 48, 34, 79, 75, 32, 31, 41].”
• Poly Kernel Inception Network for Remote Sensing Detection (2024, methodology): “PKINet-S is pretrained on ImageNet-1K [9] for 300 epochs which is consistent with previous methods [21, 32, 46] and built within the framework of Oriented RCNN [65]. mAP (07/12): VOC 2007 [13]/2012 [14] metrics.”
• YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information (2024, neutral): “[43], RTMDet [44], and PP-YOLOE [74], etc.”