An experimental comparison of classification algorithms for imbalanced credit scoring data sets

By Iain Brown, C. Mues. Expert systems with applications, 2012

718 citations recorded by the source corpus.

In this paper, we set out to compare several techniques that can be used in the analysis of imbalanced credit scoring data sets. In a credit scoring context, imbalanced data sets frequently occur as the number of defaulting loans in a portfolio is usually much lower than the number of observations that do not default. As well as using traditional classification techniques such as logistic regression, neural networks and decision trees, this paper will also explore the suitability of gradient boosting, least square support vector machines and random forests for loan default prediction.Five real-world credit scoring data sets are used to build classifiers and test their performance. In our experiments, we progressively increase class imbalance in each of these data sets by randomly under-sampling the minority class of defaulters, so as to identify to what extent the predictive power of the respective techniques is adversely affected. The performance criterion chosen to measure this effect is the area under the receiver operating characteristic curve (AUC); Friedman's statistic and Nemenyi post hoc tests are used to test for significance of AUC differences between techniques.The results from this empirical study indicate that the random forest and gradient boosting classifiers perform very well in a credit scoring context and are able to cope comparatively well with pronounced class imbalances in these data sets. We also found that, when faced with a large class imbalance, the C4.5 decision tree algorithm, quadratic discriminant analysis and k-nearest neighbours perform significantly worse than the best performing classifiers.

Subfields: Accounting, Artificial Intelligence, Finance

View Paper: https://www.semanticscholar.org/paper/76cc39ed4b40ac4220e998c0ae164925673f6185

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Accounting

In Artificial Intelligence

In Finance

Citations in the corpus

Predicting mortgage default using convolutional neural networks (2018, methodology): “RF is often credited as a very strong classifier outperforming several alternative methods (including Support Vector Machines and Neural networks) when applied to credit scoring (Brown and Mues, 2012; Kruppa et al., 2013; Lessmann et al., 2015).”
A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring (2017, methodology): “Brown and Mues (2012) compared several credit scoring algorithms using imbalanced data and suggested that RFs and gradient boosting (GB) perform comparatively well.”
A novel ensemble method for classifying imbalanced data (2015, background): “It occurs in many real-world domains [40,43,44], and has drawn a significant amount of attention in the fields of data mining and machine learning.”
Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research (2015, background): “The strongest competitor family is RF with an average rank of 14.8 (corresponding to place 12 often credited as a very strong classifier (e.g., Brown & Mues, 2012; Kruppa, 2013).”
Conditional Wasserstein GAN-based Oversampling of Tabular Data for Imbalanced Learning (2020, background): “Exactly how, and whether, class imbalance adversely a ects classi cation performance depends on the employed classi cation algorithm [9] [10], the size of the dataset [11], and on further dataset characteristics”
Inferring the outcomes of rejected loans: an application of semisupervised clustering (2019, methodology): “Hence, Bücker et al. (2013) used an improved logit model to enhance the performance of consumer credit scoring. Zeng and Zhao (2014) presented a twophase augmentation model integrated with weighted logistic regression.”
Deep learning for limit order books (2016, neutral): “Khandani et al. (2010), Butaru et al. (2015), Brown and Mues (2012), Baesens et al. (2003), Loterman et al. (2012), and Fitzpatrick and Mues (2016) use machine learning approaches for modeling consumer loan risk.”
An up-to-date comparison of state-of-the-art classification algorithms (2017, background): “Brown and Mues (2012) studied the performance of 8 classi-ﬁers, including SVM, RF, GBDT, NN, C4.5, KNN, LR, and LDA, for credit scoring, which is a binary (2-class) imbalanced classiﬁcation problem.”
Machine Learning in Banking Risk Management: A Literature Review (2019, background): “Gradient boosting and random forest classiﬁers were found to perform well in dealing with samples that exhibited a class imbalance problem (Brown and Mues 2012).”
Imbalanced enterprise credit evaluation with DTE-SBD: Decision tree ensemble based on SMOTE and bagging with differentiated sampling rates (2018, methodology): “Particularly, DT is proved to be sensitive to class imbalance [7, 11].”