Imbalanced Deep Learning by Minority Class Incremental Rectification

By Qi Dong, Shaogang Gong, Xiatian Zhu. 2018

368 citations recorded by the source corpus.

Model learning from class imbalanced training data is a long-standing and significant challenge for machine learning. In particular, existing deep learning methods consider mostly either class balanced data or moderately imbalanced data in model training, and ignore the challenge of learning from significantly imbalanced training data. To address this problem, we formulate a class imbalanced deep learning model based on batch-wise incremental minority (sparsely sampled) class rectification by hard sample mining in majority (frequently sampled) classes during model training. This model is designed to minimise the dominant effect of majority classes by discovering sparsely sampled boundaries of minority classes in an iterative batch-wise learning process. To that end, we introduce a Class Rectification Loss (CRL) function that can be deployed readily in deep network architectures. Extensive experimental evaluations are conducted on three imbalanced person attribute benchmark datasets (CelebA, X-Domain, DeepFashion) and one balanced object category benchmark dataset (CIFAR-100). These experimental results demonstrate the performance advantages and model scalability of the proposed batch-wise incremental minority class rectification model over the existing state-of-the-art models for addressing the problem of imbalanced data learning.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, background): “Huang et al. (2016) and Dong et al. (2018) demonstrate that enlarging the margin of a classiﬁer helps mitigate the class imbalance problem.”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Other than the bias related to the Softmax classifier mentioned above, another problem induced by class imbalance is under-representation of the minority [94, 95], where minority classes do not show a discernible pattern in the latent feature space [62].”
Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, background): “Imbalanced data induce severe issues in learning that are primarily attributed to gradient dominance and under-representation of the minority [44,21,98,80].”
Deep learning based brain tumor segmentation: a survey (2020, background): “to learn the morphology of the edema area, and to learn HGG instead of LGG patients) [122], [123], [49].”
Rethinking the Value of Labels for Improving Class-Imbalanced Learning (2020, background): “This posts a major challenge for modern deep learning frameworks [2,5,10,20,53], where even with specialized techniques such as data re-sampling approaches [2,5,41] or class-balanced losses [7,13,26], significant performance drops still remain under extreme class imbalance.”
Knowledge Distillation Beyond Model Compression (2020, methodology): “We follow the analysis performed in [27] whereby similar to Dong et al.[29], we employ the power law model to simulate class imbalance.”
Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks (2021, methodology): “A method that combines hard sample mining with a newly introduced class rectiﬁcation loss (CRL) function is proposed in [37].”
Delving into Deep Imbalanced Regression (2021, neutral): “Model-based solutions include re-weighting or adjusting the loss function to compensate for class imbalance (Cao et al., 2019; Cui et al., 2019; Dong et al., 2019; Huang et al., 2016; 2019), and leveraging relevant learning paradigms, including transfer learning (Yin et al., 2019), metric learning…”