Overcoming Classifier Imbalance for Long-Tail Object Detection With Balanced Group Softmax

By Yu Li, Tao Wang, Bingyi Kang, Sheng Tang, Chunfeng Wang, Jintao Li, Jiashi Feng. 2020

306 citations recorded by the source corpus.

Solving long-tail large vocabulary object detection with deep learning based models is a challenging and demanding task, which is however under-explored. In this work, we provide the first systematic analysis on the underperformance of state-of-the-art models in front of long-tail distribution. We find existing detection methods are unable to model few-shot classes when the dataset is extremely skewed, which can result in classifier imbalance in terms of parameter magnitude. Directly adapting long-tail classification models to detection frameworks can not solve this problem due to the intrinsic difference between detection and classification. In this work, we propose a novel balanced group softmax (BAGS) module for balancing the classifiers within the detection frameworks through group-wise training. It implicitly modulates the training process for the head and tail classes and ensures they are both sufficiently trained, without requiring any extra sampling for the instances from the tail classes. Extensive experiments on the very recent long-tail large vocabulary object recognition benchmark LVIS show that our proposed BAGS significantly improves the performance of detectors with various backbones and frameworks on both object detection and instance segmentation. It beats all state-of-the-art methods transferred from long-tail image classification and establishes new state-of-the-art. Code is available at https://github.com/FishYuLi/BalancedGroupSoftmax.

Subfields: Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Modern approaches for addressing long-tail data when training deep networks can be mainly divided into two groups: data re-sampling [41, 21, 62] and loss re-weighting [30, 8, 3, 54, 37, 46].”
Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation (2020, background): “With the renaissance of deep convolutional neural networks (CNNs), recent years have witnessed great progress in object recognition, including object detection [13, 22, 26, 30, 32, 34] and instance segmentation [1, 10, 16, 20, 43].”
Distribution Alignment: A Unified Framework for Long-tail Visual Recognition (2021, methodology): “The similar idea is adopted in [38, 39, 23], which develop effective strategies for long-tail object detection tasks.”
Language-Grounded Indoor 3D Semantic Segmentation in the Wild (2022, methodology): “Alternative methods have been proposed to re-balance the loss for image understanding tasks [28,21,29].”
Zero-Shot Detection via Vision and Language Knowledge Distillation (2021, methodology): “Specialized methods are proposed to address the data imbalance issue, including two-stage training [24, 26], loss re-weighting [19, 6, 5, 40, 26], and data re-sampling [30, 15, 42].”
Deep Long-Tailed Learning: A Survey (2021, methodology): “Instead of bilateral branches, BAGS [56] explored a multihead scheme to address long-tailed object detection.”
Detecting Twenty-thousand Classes using Image-level Supervision (2022, background): “Many existing works focus on handling the long-tail problem [6, 13, 27, 34, 59, 68].”
The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation (2020, methodology): “We also encourage readers to read our following work [29] that more eﬀectively calibrates the last classiﬁcation layer with a re-designed softmax module. tasks are built on region-based two-stage frameworks, e.g. , Faster R-CNN [36] and Mask R-CNN [18].”
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model (2022, background): “Besides the expensive labeling cost in this process, it often leads to a long-tailed distribution [8, 13] of object classes: detectors need to be carefully designed to avoid overfitting on frequently-occurred categories in the dataset.”
1st Place Solution of LVIS Challenge 2020: A Good Box is not a Guarantee of a Good Mask (2020, methodology): “Balanced GroupSoftmax [10] is used for balancing classifier.”
MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space (2021, background): “We note that while group-based learning has been used for improving tasks such as long-tail object detection [28], our objective and motivation are very different—we are interested in reducing the uncertainty between in- and out-ofdistribution data, rather than reducing the confusion among in-distri”
Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect (2020, unknown): “Instead of hacking the prior distribution of classes, focusing on the hard samples also alleviates the long-tailed issue, e.g. , using meta-learning to find the conditional weights for each samples [31], enhancing the samples of hard categories by group softmax [32].”
Seesaw Loss for Long-Tailed Instance Segmentation (2020, methodology): “Apart from the end-to-end training pipeline, we further explore the popular decoupling training pipeline [23, 26] on Mask R-CNN.”
Balanced Knowledge Distillation for Long-tailed Learning (2021, background): “When dealing with such imbalanced data, deep models tend to bias towards head classes, resulting in performance drop on tail classes [13, 20, 28].”
The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation (2020, methodology): “We also encourage readers to read our following work [29] that more effectively calibrates the last classification layer with a re-designed softmax module.”
Injecting Semantic Concepts into End-to-End Image Captioning (2021, unknown): “…tection [40,52], etc.”