Libra R-CNN: Towards Balanced Learning for Object Detection

By Jiangmiao Pang, Kai Chen, Jianping Shi, Huajun Feng, Wanli Ouyang, Dahua Lin. 2019

1,515 citations recorded by the source corpus.

Compared with model architectures, the training process, which is also crucial to the success of detectors, has received relatively less attention in object detection. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we propose Libra R-CNN, a simple but effective framework towards balanced learning for object detection. It integrates three novel components: IoU-balanced sampling, balanced feature pyramid, and balanced L1 loss, respectively for reducing the imbalance at sample, feature, and objective level. Benefitted from the overall balanced design, Libra R-CNN significantly improves the detection performance. Without bells and whistles, it achieves 2.5 points and 2.0 points higher Average Precision (AP) than FPN Faster R-CNN and RetinaNet respectively on MSCOCO.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
Towards Real-World Prohibited Item Detection: A Large-Scale X-ray Benchmark (2021, methodology): “Libra R-CNN [27] develops a simple and effective framework to eliminate the imbalance in the detection training process.”
Hybrid Task Cascade for Instance Segmentation (2019, background): “Mainstream object detection frameworks are often categorized into two types, single-stage detectors [29, 38, 25] and twostage detectors [39, 12, 18, 32].”
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (2020, neutral): “5 CVPR19 Libra R-CNN [22] X-101-64x4d 12 8.”
MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, methodology): “L1 loss has larger loss [23] that adopts pre-computed proposals.”
Dynamic Head: Unifying Object Detection Heads with Attentions (2021, background): “Later, [20] improved it by introducing balanced sampling and balanced feature pyramid.”
YOLOv4: Optimal Speed and Accuracy of Object Detection (2020, methodology): “The most representative two-stage object detector is the R-CNN [19] series, including fast R-CNN [18], faster R-CNN [64], R-FCN [9], and Libra R-CNN [58].”
Conditional DETR for Fast Training Convergence (2021, methodology): “Example methods include Faster R-CNN [9], SSD [26], YOLOv2 [31], YOLOv3 [32], YOLOv4 [1], RetinaNet [24], Cascade R-CNN [2], Libra R-CNN [29], TSD [35] and so on.”
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression (2019, background): “There are also several loss functions for object detection, e.g., focal loss (Lin et al. 2017), class-balanced loss (Cui et al. 2019), balanced loss for classiﬁcation and bounding box regression (Pang et al. 2019), and gradient ﬂow balancing loss (Li, Liu, and Wang 2019).”
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection (2020, neutral): “Libra R-CNN [26] X-101-64x4d 12 8.”
AugFPN: Improving Multi-Scale Feature Learning for Object Detection (2019, methodology): “As a seminal work of the two-stage detection meth-ods [10, 9, 33, 4, 21, 1, 35, 19, 20, 28], R-CNN [10] ﬁrst employs selective search [37] to generate region proposals and then reﬁnes these proposals by extracting region features through a convolutional network.”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, methodology): “Other than ResNet101 with FPN, DCN v2 [54] utilizes a ResNet101-deform v2 backbone and reports 44.0 AP.”
Dynamic Anchor Learning for Arbitrary-Oriented Object Detection (2020, background): “The most common solution is to control the ratio of candidates through a specific sampling strategy (Shrivastava, Gupta, and Girshick 2016; Pang et al. 2019).”
Binary Neural Networks: A Survey (2020, background): “Convolutional neural networks (CNNs) have been proved to be reliable in the fields of image classification [2, 3, 4, 5, 6], object detection [7, 8, 9, 10] and object recognition [11, 12, 2, 13, 14], and thus have been widely used in practice.”
Dynamic DETR: End-to-End Object Detection with Dynamic Attention (2021, background): “Later, [19] further improved it by introducing balanced feature pyramid, together with balanced sampling and and balancing loss to mitigate the adverse effects caused by the imbalance in feature level.”
PF-Net: Point Fractal Network for 3D Point Cloud Completion (2020, background): “Thanks to the architecture of FPN [17], the ﬁnal feature map is the fusion of both semantically-rich and locally-rich features [4, 25, 19, 3, 14].”