ELF: An Early-Exiting Framework for Long-Tailed Classification

By Rahul Duggal, Scott Freitas, Sunny Dhamnani, Duen Horng Chau, Jimeng Sun. 2020

30 citations recorded by the source corpus.

The natural world often follows a long-tailed data distribution where only a few classes account for most of the examples. This long-tail causes classifiers to overfit to the majority class. To mitigate this, prior solutions commonly adopt class rebalancing strategies such as data resampling and loss reshaping. However, by treating each example within a class equally, these methods fail to account for the important notion of example hardness, i.e., within each class some examples are easier to classify than others. To incorporate this notion of hardness into the learning process, we propose the EarLy-exiting Framework(ELF). During training, ELF learns to early-exit easy examples through auxiliary branches attached to a backbone network. This offers a dual benefit-(1) the neural network increasingly focuses on hard examples, since they contribute more to the overall network loss; and (2) it frees up additional model capacity to distinguish difficult examples. Experimental results on two large-scale datasets, ImageNet LT and iNaturalist'18, demonstrate that ELF can improve state-of-the-art accuracy by more than 3 percent. This comes with the additional benefit of reducing up to 20 percent of inference time FLOPS. ELF is complementary to prior work and can naturally integrate with a variety of existing methods to tackle the challenge of long-tailed distributions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Jiaya Jia
5. Shu Liu
6. Le Yang
7. Yizeng Han
8. Zhisheng Zhong

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Gao Huang
3. Jiaya Jia
4. Luc Van Gool
5. Shu Liu
6. Ming-Hsuan Yang
7. Yulin Wang
8. Xiaogang Wang

In Radiology, Nuclear Medicine and Imaging

1. Jiequan Cui
2. Jiaya Jia
3. Shu Liu
4. Zhisheng Zhong

In Representation & Self-Supervised Learning

1. Gao Huang
2. Jiequan Cui
3. Jiaya Jia
4. Shiji Song
5. Le Yang
6. Bei Yu
7. Shu Liu
8. Yulin Wang

In Efficiency & Compression

1. Shiji Song
2. Gao Huang
3. Yulin Wang
4. Le Yang
5. Yizeng Han
6. Honghui Wang
7. Jiequan Cui
8. Jiaya Jia

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, methodology): “For example, multi-exit models can be used to: 1) alleviate the over-thinking issue while reducing the overall computation [50], [266]; 2) perform long-tailed classification [267] by inducing early exiting in the training stage; and 3) improve the model robustness [268].”
• Parametric Contrastive Learning (2021, background): “However the center learning also needs to be balanced, which has been explored in [1, 25, 15, 20, 8, 41, 38, 28, 49, 14, 46, 18, 57].”