Survey on deep learning with class imbalance

By Justin M. Johnson, Taghi M. Khoshgoftaar. Journal of Big Data, 2019

2,770 citations recorded by the source corpus.

The purpose of this study is to examine existing deep learning techniques for addressing class imbalanced data. Effective classification with imbalanced data is an important area of research, as high class imbalance is naturally inherent in many real-world applications, e.g., fraud detection and cancer detection. Moreover, highly imbalanced data poses added difficulty, as most learners will exhibit bias towards the majority class, and in extreme cases, may ignore the minority class altogether. Class imbalance has been studied thoroughly over the last two decades using traditional machine learning models, i.e. non-deep learning. Despite recent advances in deep learning, along with its increasing popularity, very little empirical work in the area of deep learning with class imbalance exists. Having achieved record-breaking performance results in several complex domains, investigating the use of deep neural networks for problems containing high levels of class imbalance is of great interest. Available studies regarding class imbalance and deep learning are surveyed in order to better understand the efficacy of deep learning when applied to class imbalanced data. This survey discusses the implementation details and experimental results for each study, and offers additional insight into their strengths and weaknesses. Several areas of focus include: data complexity, architectures tested, performance interpretation, ease of use, big data application, and generalization to other domains. We have found that research in this area is very limited, that most existing work focuses on computer vision tasks with convolutional neural networks, and that the effects of big data are rarely considered. Several traditional methods for class imbalance, e.g. data sampling and cost-sensitive learning, prove to be applicable in deep learning, while more advanced methods that exploit neural network feature learning abilities show promising results. The survey concludes with a discussion that highlights various gaps in deep learning from class imbalanced data for the purpose of guiding future research.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/f9163156eeba67762a7441db48fe6720106137cd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Media Technology

Citations in the corpus

A Survey on Deep Semi-Supervised Learning (2021, background): “When the training data is highly imbalanced, most learning frameworks will show bias towards the majority class, and in some extreme cases, may completely ignore the minority class [126], as a result, the efficiency of predictive models will be significantly affected.”
On Cross-Dataset Generalization in Automatic Detection of Online Abuse (2020, background): “In practice, simple re-sampling techniques, such as down-sampling of over-represented classes, often improve the overall performance of the classifier (Johnson and Khoshgoftaar, 2019).”
DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data (2021, methodology): “attempts have been made to extend oversampling methods, such as SMOTE, to deep learning models, although the results have been mixed [84]–[86].”
Token-level Adaptive Training for Neural Machine Translation (2020, neutral): “This problem can be observed in various tasks (Wei et al., 2013; Johnson and Khoshgoftaar, 2019).”
Disentangling Label Distribution for Long-tailed Visual Recognition (2020, background): “Note that the concept of the PC strategy is not entirely new since [40, 7, 26] previously covered as a different form of multiplying pt(y)/ps(y) to the output probability.”
Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis (2020, background): “It is worth mentioning that most of the existing work dealing with imbalanced data in the context of deep learning focuses on cases where it is related to the target classes (14, 15).”
Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-rays (2020, methodology): “We used model checkpoints to store the best model weights for further analysis.”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, background): “However, different data sampling strategies have been considered for imbalanced data, e.g., oversampling and undersampling [82].”
A Comparison of Undersampling, Oversampling, and SMOTE Methods for Dealing with Imbalanced Classification in Educational Data Mining (2023, methodology): “The data-level approach to imbalance learning aims to create a new dataset with an equal ratio between classes of the target variable to reduce the bias of the predictive algorithm [5,17].”
Knowledge Distillation Beyond Model Compression (2020, background): “class imbalance exists within the training data, DNNs tend to over-classify the majority group due to its increased prior probability [9].”
ImageNet-21K Pretraining for the Masses (2021, background): “In addition, due to the semantic structure, multi-label training is hindered by a large class imbalance [21] - on average, classes from a lower hierarchy will appear far more frequent than classes from a higher hierarchy.”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “As for class imbalance, when the training data is highly imbalanced, the trained model will show bias towards the majority classes, and may completely ignore the minority classes in some extreme cases [134].”
Deep Learning for Medical Anomaly Detection – A Survey (2020, background): “The most common approaches to address class imbalance in medical anomaly detection has been data re-sampling (under- or oversampling) and cost-sensitive training where more weight in the loss is assigned to the minority class [64].”