Bounding Box Regression With Uncertainty for Accurate Object Detection

By Yihui He, Chenchen Zhu, Jianren Wang, Marios Savvides, X. Zhang. 2018

546 citations recorded by the source corpus.

Large-scale object detection datasets (e.g., MS-COCO) try to define the ground truth bounding boxes as clear as possible. However, we observe that ambiguities are still introduced when labeling the bounding boxes. In this paper, we propose a novel bounding box regression loss for learning bounding box transformation and localization variance together. Our loss greatly improves the localization accuracies of various architectures with nearly no additional computation. The learned localization variance allows us to merge neighboring bounding boxes during non-maximum suppression (NMS), which further improves the localization performance. On MS-COCO, we boost the Average Precision (AP) of VGG-16 Faster R-CNN from 23.6% to 29.1%. More importantly, for ResNet-50-FPN Mask R-CNN, our method improves the AP and AP90 by 1.8% and 6.2% respectively, which significantly outperforms previous state-of-the-art bounding box refinement methods. Our code and models are available at github.com/yihui-he/KL-Loss

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (2020, methodology): “Recently, Gaussian assumption [10, 4] is adopted to learn the uncertainty by introducing a predicted variance.”
Feature Selective Anchor-Free Module for Single-Shot Object Detection (2019, background): “He et al [14] modeled the bounding box as Gaussian distribution for improved localization.”
Soft Anchor-Point Object Detection (2019, background): “Compared to conventional anchor-based approaches [23,18,22,3,24,4,14,15,31,1,9], anchor-free detectors have a few advantages in general: 1) no manual tuning of hyperparameters for the anchor configuration; 2) usually simpler architecture of detection head; 3) less training memory cost.”
Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation (2020, background): “Recently, Adaptive NMS [51] and Softer-NMS [52] are proposed to respectively study proper threshold and weighted average strategies.”
Human Pose Regression with Residual Log-likelihood Estimation (2021, background): “For example, in some works of object detection [16, 26, 25] and dense correspondences [37], the density is assumed to be a Gaussian distribution.”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, unknown): “into single-stage [35, 42, 40, 29, 27, 4] and two-stage methods [18, 43, 17, 41, 8, 21].”
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression (2019, background): “Soft-NMS (Bodla et al. 2017) penalizes the detection score of neighbors by a continuous function w.r.t. IoU, yielding softer and more robust suppression than original NMS. IoU-Net (Jiang et al. 2018) introduces a new network branch to predict the localization conﬁdence to guide NMS. Recently, adapti”
Energy-Based Models for Deep Probabilistic Regression (2020, background): “detection [13, 19] and optical flow estimation [15, 22], usually aiming to not only achieve accurate predictions, but also to provide an estimate of aleatoric uncertainty [24, 17].”
Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training (2020, background): “calization accuracy, which leads to noisy ranking and limited performance. Therefore, IoU-Net [19] adopts an extra branch for predicting IoU scores and rening the classication condence. Softer NMS [17] devises an KL loss to model the variance of bounding box regression directly, and uses that for vo”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “To date, there have been number of studies addressed uncertainty in deep learning algorithms for various applications including but not limited to image/video retrieval [399], [400], depth estimation [401], [402], object detection [403], [404], [405], semantic segmentation and scene understanding [4”
Imbalance Problems in Object Detection: A Review (2019, methodology): “Another approach, Kullback-Leibler Loss (KL Loss) [53], is driven by the fact that the ground truth boxes can be ambiguous in some cases due to e.”
Learning to Rank Proposals for Object Detection (2019, methodology): “For the comparison with Softer-NMS [14], its map result achieves 39.2 AP on COCO val2017 by lengthening the learning period.”
Detection in Crowded Scenes: One Proposal, Multiple Predictions (2020, background): “Previous works have tried to address this issue from different perspectives, such as sophisticated NMS [1, 16, 23, 18, 25, 17], new loss functions [44, 40], re-scoring [19], part-based detectors [44, 38, 46, 3].”
SOLOv2: Dynamic and Fast Instance Segmentation (2020, methodology): “In [13], the authors use KL-Divergence and reﬂected it in the reﬁnement of coordinates in the NMS process.”
A Survey on Few-Shot Class-Incremental Learning (2023, background): “However, they do not successfully explain why their estimation of the uncertainty of bounding-box localization surpasses a Gaussian-based uncertainty estimation (He et al., 2019).”