Revisiting the Sibling Head in Object Detector

By Guanglu Song, Yu Liu, Xiaogang Wang. 2020

415 citations recorded by the source corpus.

The "shared head for classification and localization'' (sibling head), firstly denominated in Fast RCNN, has been leading the fashion of the object detection community in the past five years. This paper provides the observation that the spatial misalignment between the two object functions in the sibling head can considerably hurt the training process, but this misalignment can be resolved by a very simple operator called task-aware spatial disentanglement (TSD). Considering the classification and regression, TSD decouples them from the spatial dimension by generating two disentangled proposals for them, which are estimated by the shared proposal. This is inspired by the natural insight that for one instance, the features in some salient area may have rich information for classification while these around the boundary may be good at bounding box regression. Surprisingly, this simple design can boost all backbones and models on both MS COCO and Google OpenImage consistently by ~3% mAP. Further, we propose a progressive constraint to enlarge the performance margin between the disentangled and the shared proposals, and gain ~1% more mAP. We show the TSD breaks through the upper bound of nowadays single-model detector by a large margin (mAP 49.4 with ResNet-101, 51.2 with SENet154), and is the core model of our 1st place solution on the Google OpenImage Challenge 2019.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (2020, neutral): “-101-DCN 24 X 11.8 45.0 66.1 49.0 26.6 48.6 57.5 ICCV19 TridentNet [16] R-101 24 X 2.7 42.7 63.6 46.5 23.9 46.6 56.6 ICCV19 TridentNet [16] R-101-DCN 36 X 1.3 46.8 67.6 51.5 28.0 51.2 60.5 ICCV19 TSD [25] R-101 20 1.1 43.2 64.0 46.9 24.0 46.3 55.8 CVPR20 one-stage: CornerNet [14] HG-104 200 X 3.1 40”
Conditional DETR for Fast Training Convergence (2021, methodology): “Example methods include Faster R-CNN [9], SSD [26], YOLOv2 [31], YOLOv3 [32], YOLOv4 [1], RetinaNet [24], Cascade R-CNN [2], Libra R-CNN [29], TSD [35] and so on.”
UP-DETR: Unsupervised Pre-training for Object Detection with Transformers (2020, background): “Object detection is the coupling of object classification and localization, where these two tasks always have different feature preferences [45, 40, 35].”
Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression (2021, background): “Two-stage anchor-based detectors (e.g., R-CNN series [11, 31, 14, 3], HTC [5], and TSD [33]) are firstly proposed in object detection tasks, which are composed of region proposal networks (RPNs) and classifiers.”
Probabilistic two-stage detection (2021, background): “Also, one-stage detectors only use the local feature of the positive cell for regression and classiﬁcation, which is sometimes misaligned with the object (Chen et al., 2019c; Song et al., 2020).”
TOOD: Task-aligned One-stage Object Detection (2021, background): “design, the task-interactive features inevitably introduce a certa n level of feature conflicts between two different tasks, which have also been discussed in [26, 28].”
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection (2020, neutral): “TSD [32] R-101 20 1.”
DETRs with Collaborative Hybrid Assignments Training (2022, background): “a series of variants [30, 36, 41] such as ATSS [39], RetinaNet [20], FCOS [31], and PAA [16] lead to the significant breakthrough of object detection task.”
Dynamic Anchor Learning for Arbitrary-Oriented Object Detection (2020, background): “Some researches (Zhang et al. 2019; Song, Liu, and Wang 2020) have reported that the discriminative features required to localize objects are not evenly distributed on GT, especially for objects with a wide variety of orientations and aspect ratios.”
A Comprehensive Review of YOLO: From YOLOv1 to YOLOv8 and Beyond (2023, background): “In [82, 83], it was shown that there could be a misalignment between the classiﬁcation conﬁdence and localization accuracy.”
Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020, neutral): “4 TSD (Song et al., 2020) SENet154 + DCN X 51.”
Simple Multi-dataset Detection (2021, result): “On OpenIm-ages, our result matches the best single model in the Open-Images 2019 Challenge, TSD [36], with a comparable back-bone (SENet154-DCN [16] of TSD).”
Learning Spatio-Temporal Transformer for Visual Tracking (2021, background): “As pointed out by recent works [7, 43], jointly learning of localization and classiﬁcation may cause sub-optimal solutions for both tasks, and it is helpful to decouple localization and classiﬁcation.”
Rethinking Transformer-based Set Prediction for Object Detection (2020, methodology): “We compare TSP-RCNN with multiple one-stage and two-stage object detection models [26, 31, 1, 28, 19, 40, 30, 3, 16, 35, 38] that also use ResNet-101 backbone or its deformable convolution network (DCN) [41] variant in Table 4.”
Seesaw Loss for Long-Tailed Instance Segmentation (2020, methodology): “TSD. TSD [43] is used to replace the box heads in all three stages in HTC-Lite.”
A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS (2023, background): “In [94, 95], it was shown that there could be a misalignment between the classification confidence and localization accuracy.”