VarifocalNet: An IoU-aware Dense Object Detector

By Haoyang Zhang, Feras Dayoub, Niko Sünderhauf. Computer Vision and Pattern Recognition, 2020

1,032 citations recorded by the source corpus.

Accurately ranking the vast number of candidate detections is crucial for dense object detectors to achieve high performance. Prior work uses the classification score or a combination of classification and predicted localization scores to rank candidates. However, neither option results in a reliable ranking, thus degrading detection performance. In this paper, we propose to learn an Iou-Aware Classification Score (IACS) as a joint representation of object presence confidence and localization accuracy. We show that dense object detectors can achieve a more accurate ranking of candidate detections based on the IACS. We design a new loss function, named Varifocal Loss, to train a dense object detector to predict the IACS, and propose a new star-shaped bounding box feature representation for IACS prediction and bounding box refinement. Combining these two new components and a bounding box refinement branch, we build an IoU-aware dense object detector based on the FCOS+ATSS architecture, that we call VarifocalNet or VFNet for short. Extensive experiments on MS COCO show that our VFNet consistently surpasses the strong baseline by ~2.0 AP with different backbones. Our best model VFNet-X-1200 with Res2Net-101-DCN achieves a single-model single-scale AP of 55.1 on COCO test-dev, which is state-of-the-art among various object detectors. Code is available at: https://github.com/hyz-xmaster/VarifocalNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/14c3510e4f4b370d5cd0420037406024533f4b6f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Comprehensive Review of YOLO: From YOLOv1 to YOLOv8 and Beyond (2023, methodology): “They used a classiﬁcation VariFocal loss [89] and a SIoU [90]/GIoU [91] regression loss.”
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection (2020, methodology): “After that, IoU is further explored and proved to be effective in IoU-Net [13], IoU-aware [36], PAA [14], GFLV1 [18] and VFNet [39].”
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios (2021, neutral): “2) two-stage detectors: VFNet [59], CenterNet2 [62].”
DETRs Beat YOLOs on Real-time Object Detection (2023, methodology): “We introduce the IoU score into objective function of the classification branch (similar to VFL [41]) to realize the consistency constraint on the classification and localization of positive samples.”
A Review on Deep Learning in UAV Remote Sensing (2021, methodology): “The Libra-RCNN [147], ATSS [7], Guided Anchoring [61], FSAF [216], PAA [145], GFL [65], PISA [153] and VFNet [191] detectors explore di ff erent sampling strategies and new loss metrics to improve the quality of selected positive samples and reduce the weight of the large negative samples.”
3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding (2022, neutral): “Tecent Youtu Lab (1st place) and Angelo Menezes (3rd place) are using single-stage detectors (VarifocalNet [79] and FCOS [26]) while NUS&A*STAR (2nd place) is using two-stage detector (Faster R-CNN [20]).”
Simple Multi-dataset Detection (2021, background): “This fragmentation brought rapid progress in object detection [5, 10, 20, 31, 39, 45] and instance segmentation [14], but comes with a drawback.”
YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications (2022, methodology): “We choose VFL as the classification loss function.”
SwinTrack: A Simple and Strong Baseline for Transformer Tracking (2021, methodology): “In classification branch, we employ the IoU-aware classification score as the training target and the varifocal loss [45] as the training loss function.”
TOOD: Task-aligned One-stage Object Detection (2021, neutral): “0 VFNet [29]† CVPR21 ResNeXt-101-64×4d-DCN 49.”
Slicing Aided Hyper Inference and Fine-Tuning for Small Object Detection (2022, methodology): “In this field, adoption of deep learning architectures has resulted in highly accurate methods such as Faster R-CNN [1], RetinaNet [2], that are further developed as Cascade R-CNN [3], VarifocalNet [4], and variants.”
YOLO-based Object Detection Models: A Review and its Applications (2024, neutral): “YOLOv6 provides two loss functions: Varifocal loss (VFL) [83] and distribution focal loss (DFL) [84].”