Acquisition of Localization Confidence for Accurate Object Detection

By Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, Yuning Jiang. 2018

947 citations recorded by the source corpus.

Modern CNN-based object detectors rely on bounding box regression and non-maximum suppression to localize objects. While the probabilities for class labels naturally reflect classification confidence, localization confidence is absent. This makes properly localized bounding boxes degenerate during iterative regression or even suppressed during NMS. In the paper we propose IoU-Net learning to predict the IoU between each detected bounding box and the matched ground-truth. The network acquires this confidence of localization, which improves the NMS procedure by preserving accurately localized bounding boxes. Furthermore, an optimization-based bounding box refinement method is proposed, where the predicted IoU is formulated as the objective. Extensive experiments on the MS-COCO dataset show the effectiveness of IoU-Net, as well as its compatibility with and adaptivity to several state-of-the-art object detectors.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “But unlike RoI Align used in Mask-RCNN [13], they use Precise RoI Pooling [16] to extract features for each proposal.”
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression (2019, methodology): “Similarly, for the object detection task, recent works [8, 24] have attempted to directly or indirectly incorporate IoU to better perform bounding box regression.”
Learning Open-World Object Proposals without Learning to Classify (2021, methodology): “Compared to IoU-Net [16] which requires manual proposal generation for IoU training, OLN directly computes the IoU targets from the OLN-RPN proposals and ground-truth boxes, thus greatly saving computation costs.”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “IoU-Net [20] performs progressive bounding box refinement (even though not presenting a cascade structure explicitly).”
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (2020, methodology): “the quality estimation is combined (usually multiplied) with classification confidence as final scores [12, 11, 26, 29, 35] f r the rank process of Non-Maximum Suppression (NMS) during inference.”
CornerNet: Detecting Objects as Paired Keypoints (2018, neutral): “…of the bounding boxes predicted by CornerNet, we evaluate the performance of CornerNet at multiple IoU thresholds, and compare the results with other state-of-the-art detectors, including RetinaNet (Lin et al., 2017), Cascade R-CNN (Cai and Vasconcelos, 2017) and IoU-Net (Jiang et al., 2018).”
Mask Scoring R-CNN (2019, methodology): “Since there is no method works on predicting MaskIoU before, we refer to a previous work [19] on predicting bounding box IoU.”
Libra R-CNN: Towards Balanced Learning for Object Detection (2019, background): “Recently, UnitBox [34] and IoUNet [15] introduced some new objective functions related to IoU, to promote the localization accuracy.”
Bottom-Up Object Detection by Grouping Extreme and Center Points (2019, background): “The detection-by-classiﬁcation idea is intuitive and keeps the best performance so far [6,7, 19,20,24,27,37,45,46,54].”
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, methodology): “As motivated by (Jiang et al. 2018; Li et al. 2019; Shi et al. 2020a,b), the box regression branch predicts the residue from 3D region proposals to the ground truth boxes, and the conﬁdence branch predicts the IoU-related conﬁdence score.”
Adaptive NMS: Refining Pedestrian Detection in a Crowd (2019, methodology): “[41] and [17] replace the classification scores of proposals used in the NMS process with learned localization confidences to guide NMS to preserve more accurately localized bounding boxes.”
SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines (2019, methodology): “Inspired by DCF and IoU-Net (Jiang et al. 2018), ATOM (Danelljan et al. 2019) tracks target by sequential classiﬁcation and estimation.”
CentripetalNet: Pursuing High-Quality Keypoint Pairs for Object Detection (2020, positive): “Other works cooperating with anchor-based detectors are proposed to deal with different issues, such as improving anchor selection procedure [33], refining feature learning process [39, 18], optimizing location prediction method [24], and improving the loss function [30, 16].”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, background): “Several recent works have extended this framework by, for example, integrating pyramid representations [31, 28, 44, 5], extending to multi-stage detection [16, 2, 6, 24] and integrating a mask branch [19, 23, 34].”
FoveaBox: Beyound Anchor-Based Object Detection (2019, background): “including FPN [21], Mask R-CNN [8] and IoU-Net [38].”
Recent Advances in Deep Learning for Object Detection (2019, methodology): “[123] presented Precise ROI Pooing (PrROI Pooling), which 1014 avoided any quantization of coordinates and had a continuous gra1015 dient on bounding box coordinates.”