Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

By Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, Stella X. Yu. 2020

500 citations recorded by the source corpus.

Natural data are often long-tail distributed over semantic classes. Existing recognition methods tend to focus on tail performance gain, often at the expense of head performance loss from increased classifier variance. The low tail performance manifests itself in large inter-class confusion and high classifier variance. We aim to reduce both the bias and the variance of a long-tailed classifier by RoutIng Diverse Experts (RIDE). It has three components: 1) a shared architecture for multiple classifiers (experts); 2) a distribution-aware diversity loss that encourages more diverse decisions for classes with fewer training instances; and 3) an expert routing module that dynamically assigns more ambiguous instances to additional experts. With on-par computational complexity, RIDE significantly outperforms the state-of-the-art methods by 5% to 7% on all the benchmarks including CIFAR100-LT, ImageNet-LT and iNaturalist. RIDE is also a universal framework that can be applied to different backbone networks and integrated into various long-tailed algorithms and training mechanisms for consistent performance gains.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiaya Jia
2. Shu Liu
3. Andrew D. Bagdanov
4. Xialei Liu
5. Yi-Zhe Song
6. Xiu-Shen Wei
7. Jian Yang
8. Jianxin Wu

In Radiology, Nuclear Medicine and Imaging

1. Jiaya Jia
2. Shu Liu
3. Jiequan Cui
4. Zhisheng Zhong
5. Mingg-Ming Cheng
6. Yi-Zhe Song
7. Andrew D. Bagdanov
8. Xialei Liu

Citations in the corpus

• Long-Tailed Class Incremental Learning (2022, background): “Learning from Multiple Experts (LFME) [42] and Routing Diverse Experts (RIDE) [39] adopt the same idea of ensemble learning to aggregate knowledge from multiple experts.”
• Parametric Contrastive Learning (2021, methodology): “state-of-the-arts of Decouple [28], Balanced Softmax [38] and RIDE [49] as mentioned in Section 4.”
• Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “State-of-the-art results on these datasets are 72.6% [191], 78.5% [27], 80.5% [192], respectively, which reveals the substantial room for further improvement.”