Balanced Activation for Long-tailed Visual Recognition

By Jiawei Ren, Cunjun Yu, Zhongang Cai, Haiyu Zhao. 2020

Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this report, we introduce Balanced Activation (Balanced Softmax and Balanced Sigmoid), an elegant unbiased, and simple extension of Sigmoid and Softmax activation function, to accommodate the label distribution shift between training and testing in object detection. We derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In our experiments, we demonstrate that Balanced Activation generally provides ~3% gain in terms of mAP on LVIS-1.0 and outperforms the current state-of-the-art methods without introducing any extra parameters.

Subfields: Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Quoc V. Le
3. Ming-Hsuan Yang
4. Trevor Darrell
5. Xiaogang Wang
6. Alan L. Yuille
7. Dacheng Tao
8. Jitendra Malik

In Optimization & Training

1. Quoc V. Le
2. Ekin Dogus Cubuk
3. Barret Zoph
4. Tsung-Yi Lin
5. Golnaz Ghiasi
6. A. Srinivas
7. Yin Cui
8. Frank Hutter

Citations in the corpus

• Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Modern approaches for addressing long-tail data when training deep networks can be mainly divided into two groups: data re-sampling [41, 21, 62] and loss re-weighting [30, 8, 3, 54, 37, 46].”