Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance

By Maciej A. Mazurowski, Piotr A. Habas, J. Zurada, Joseph Y. Lo, Jay A. Baker, Georgia D. Tourassi. 2008

813 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Health Information Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Francisco Herrera
2. Salvador Garcia
3. Alberto Fernández
4. Victoria López
5. Siti Mariyam Hj. Shamsuddin
6. Xiatian Zhu
7. Aida Ali
8. Anca L. Ralescu

In Health Information Management

1. Qeethara Al-Shayea
2. Alberto Fernández
3. Shaogang Gong
4. Francisco Herrera
5. Maciej A. Mazurowski
6. Atsuto Maki
7. Xiatian Zhu
8. Mateusz Buda

Citations in the corpus

• A systematic study of the class imbalance problem in convolutional neural networks (2017, methodology): “It has been established that class imbalance can have significant detrimental effect on training of traditional classifiers [17] including multi-layer perceptrons [18].”
• Hidden stratification causes clinically meaningful failures in machine learning for medical imaging (2019, background): “served stratification and dataset imbalance in medical machine learning often center on gathering more data on underperforming subsets, either via additional labelling, selective data augmentation, or oversampling [22].”
• SMOTE-RSB*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using SMOTE and rough sets theory (2012, background): “There are many situations in which imbalance occurs between classes, such as satellite image classification [38], risk management [26], medical applications [29], and so on.”
• Imbalanced Deep Learning by Minority Class Incremental Rectification (2018, background): “Third, directly incorporating existing imbalanced data learning algorithms into a deep learning framework does not provide effective solutions [26,27,28].”
• Artificial Neural Networks in Medical Diagnosis (2024, background): “Mazurowski, Habas, Zurada, Lo, Baker and Tourassi [11] investigated the effect of class imbalance in training data when developing neural network classifiers for computer aided medical diagnosis.”
• Artificial neural networks in medical diagnosis (2013, background): “A decrease in the classification performance of the network is observed for imbalanced databases (those with a different number of cases for each class) (Mazurowski et al. 2008).”
• Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks (2021, background): “Data distribution imbalance is predominant in many real-world classification tasks, such as fault diagnosis [2], [3], network intrusion detection [4], medical diagnosis [5], [6], electricity pilferage [7] and fraudulent transactions [8], among others.”
• An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics (2013, background): “As some examples, we could mention very high resolution airbourne imagery [31], forecasting of ozone levels [125], face recognition [78], and especially medical diagnosis [11,86,91,93,132].”
• Deep Learning Applications in Medical Image Analysis (2018, background): “Data or class imbalance in the training set is also a significant issue in medical image analysis [94].”
• DEEP NEURAL (positive): “The importance of balancing datasets is highlighted by the fact that deep neural networks may be most valuable in the work up of rare or challenging diseases, which practitioners at a common skill level may fail to recognize or misinterpret [7].”