BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition

By Boyan Zhou, Quan Cui, Xiu-Shen Wei, Zhao-Min Chen. 2019

993 citations recorded by the source corpus.

Our work focuses on tackling the challenging but natural visual recognition task of long-tailed data distribution (i.e., a few classes occupy most of the data, while most classes have rarely few samples). In the literature, class re-balancing strategies (e.g., re-weighting and re-sampling) are the prominent and effective methods proposed to alleviate the extreme imbalance for dealing with long-tailed problems. In this paper, we firstly discover that these re-balancing methods achieving satisfactory recognition accuracy owe to that they could significantly promote the classifier learning of deep networks. However, at the same time, they will unexpectedly damage the representative ability of the learned deep features to some extent. Therefore, we propose a unified Bilateral-Branch Network (BBN) to take care of both representation learning and classifier learning simultaneously, where each branch does perform its own duty separately. In particular, our BBN model is further equipped with a novel cumulative learning strategy, which is designed to first learn the universal patterns and then pay attention to the tail data gradually. Extensive experiments on four benchmark datasets, including the large-scale iNaturalist ones, justify that the proposed BBN can significantly outperform state-of-the-art methods. Furthermore, validation experiments can demonstrate both our preliminary discovery and effectiveness of tailored designs in BBN for long-tailed problems. Our method won the first place in the iNaturalist 2019 large scale species classification competition, and our code is open-source and available at https://github.com/Megvii-Nanjing/BBN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Recent work [9, 3, 33, 71, 37] has pointed out the effectiveness of two-stage training strategies by separating the feature learning and the re-balancing stage, as end-to-end training with re-balancing strategies could be detrimental to feature learning.”
Neural Prototype Trees for Interpretable Fine-grained Image Recognition (2020, methodology): “For CUB, ResNet50 is pretrained on iNaturalist2017 [49], containing plants and animals and therefore a suitable source domain [32], using the backbone of [59].”
Long-Tailed Class Incremental Learning (2022, background): “Bi-lateral Branch Networks (BBNs) [51] use two network branches, a conventional learning branch and a re-balancing branch, to address the longtailed recognition problem [51].”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, background): “Directly training a system over all classes is prone to over-fit and can be biased towards the data-rich categories [6, 12, 25, 69, 72].”
Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia (2020, methodology): “Then, inspired by the class-resampling strategy in [45], we define the weights [W covid small,W covid large ,W cap small,W cap large] for 4 groups as [N covid large /N covid small, 1, 1, N cap small/N cap large].”
Improving Calibration for Long-Tailed Recognition (2020, methodology): “[39], we conduct experiments with IF 100, 50, and 10.”
Remix: Rebalanced Mixup (2020, background): “9.86 Remix-DRW 62.95 67.76 75.49 79.43 62.82 67.56 76.55 79.36 which considers the trade-o between the class margins. (7) Mixup: Each batch of training samples are generated according to [37] (8) BBN [38] and LWS [17]: Two state-of-the-arts methods. We directly copy the results from the original pap”
Exploring Balanced Feature Spaces for Representation Learning (2021, neutral): “CB-Focal (Cui et al., 2019) 61.1 LDAM (Cao et al., 2019) 64.6 LDAM+DRW (Cao et al., 2019) 68.0 cRT (Kang et al., 2020) 65.2 τ -norm (Kang et al., 2020) 65.6 BBN (Zhou et al., 2020) 66.3 thus less balanced as demonstrated in Sec.”
Long-tailed Recognition by Routing Diverse Distribution-Aware Experts (2020, neutral): “Compared with BBN (Zhou et al., 2020) and LFME (Xiang et al., 2020), which also contain multiple experts (or branches), RIDE (2 experts) outperforms them by a large margin with fewer GFlops.”
DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data (2021, background): “[81] proposed a cumulative learning scheme from global data properties down to class-based features.”
Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification (2021, methodology): “To realize the idea that better features ease classifier learning and consequently result in more generalizable classifiers, we follow a curriculum [37] to adjust the weightings of these two branches during the training phase.”
ELF: An Early-Exiting Framework for Long-Tailed Classification (2020, methodology): “This imbalance problem has traditionally been solved by resampling the data (undersampling, oversampling) [1, 2, 3, 4, 5], or reshaping the loss function (loss reweighting, regularization) [6, 7].”
Rethinking the Value of Labels for Improving Class-Imbalanced Learning (2020, positive): “Recent studies [25, 59] also find that decoupling the representation and classifier leads to better long-tailed learning results.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “More research is needed in areas such as domain adaptation [207], [208], [209], long-tailed distributions [210], [211], open world settings [212], scale variations [2], fine-grained video understanding [213], [214], knowledge transfer, and resource constrained embedded deployment, to name a few.”
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective (2022, background): “Recent works [18,48] revealed that the performance of a classification model is closely related to the quality of its classifier.”
Parametric Contrastive Learning (2021, methodology): “[58] proposed one cumulative learning strategy, with which they bridge representation learning and classifier re-balancing.”
Balanced Knowledge Distillation for Long-tailed Learning (2021, methodology): “Although effective, such re-balancing strategies will encourage excessive focus on tail classes and damage the overall representation learning to some extent [5, 56].”