Comprehensible credit scoring models using rule extraction from support vector machines

By David Martens, Bart Baesens, Tony Van Gestel, Jan Vanthienen. 2007

475 citations recorded by the source corpus.

In recent years, support vector machines (SVMs) were successfully applied to a wide range of applications. However, since the classifier is described as a complex mathematical function, it is rather incomprehensible for humans. This opacity property prevents them from being used in many real-life applications where both accuracy and comprehensibility are required, such as medical diagnosis and credit risk evaluation. To overcome this limitation, rules can be extracted from the trained SVM that are interpretable by humans and keep as much of the accuracy of the SVM as possible. In this paper, we will provide an overview of the recently proposed rule extraction techniques for SVMs and introduce two others taken from the artificial neural networks domain, being Trepan and G-REX. The described techniques are compared using publicly available datasets, such as Ripley’s synthetic dataset and the multi-class iris dataset. We will also look at medical diagnosis and credit scoring where comprehensibility is a key requirement and even a regulatory recommendation. Our experiments show that the SVM rule extraction techniques lose only a small percentage in performance compared to SVMs and therefore rank at the top of comprehensible classification techniques.

Subfields: Accounting, Artificial Intelligence, Health Information Management, Optimization & Training, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Accounting

In Artificial Intelligence

In Health Information Management

In Optimization & Training

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “In Reference [76] the authors propose a qualitative comparison of the explanations returned by techniques for extraction of rules from SVM black boxes (e.”
Opening black box Data Mining models using Sensitivity Analysis (2011, background): “However, such extraction is often based on a process that simpliﬁes the model complexity and may lead to rules that do not accurately represent the original model.”
Interval Coded Scoring: a toolbox for interpretable scoring systems (2018, background): “Further examples includes NNs for severity assessment of breast cancer (Maity and Das, 2017) and SVMs for detection of mild cognitive impairment and Alzheimer disease from multimodal brain imaging (Shen et al., 2014).”
Strongly agree or strongly disagree?: Rating features in Support Vector Machines (2016, methodology): “…Romero Morales (2013) for a recent review on Mathematical Optimization and the SVM, and Bertsimas et al. (2008), Brooks (2011), Cecchini et al. (2010), Chaovalitwongse et al. (2008), Guyon et al. (2002), Martens et al. (2007), Romero Morales and Wang (2010) for successful applications of the SVM.”
Methods and Models for Interpretable Linear Classification (2014, methodology): “The interpretability of black-box models is mainly improved by auxiliary methods that extract rules and prototype examples to illustrate the relationship between input variables and the predicted outcome [78, 111, 40, 76, 11].”
Explaining Explanations in AI (2018, background): “Historically, much work has focused on global approximations of models (e.g. Craven and Shavlik (1996); Martens et al. (2007); Sanchez et al. (2015), including approaches based on clustering (Chen et al., 2016), integer programming (Zeng et al., 2017), and rule lists (Wang and Rudin, 2015)).”
Optimal randomized classification trees (2021, neutral): “Extracting knowledge from data is a crucial task in Statistics and Machine Learning, which has applications in areas such as Biomedicine [1, 2], Criminal Justice [3, 4, 5], Fraud Detection [6], Privacy Protection [7], Health Care [8, 9, 10, 11], Risk Management [12, 13], Social Networks [14], and Co”
Efficient Search for Diverse Coherent Explanations (2019, background): “These approximate models are either fitted globally (Craven and Shavlik, 1996; Martens et al., 2007; Sanchez et al., 2015) over the entire space of valid datapoints, or as a local approximation (Lundberg and Lee, 2017; Montavon et al., 2017; Ribeiro et al., 2016; Shrikumar et al., 2016) that only…”
A model explanation system (2016, background): “Analogous scenarios occur in computer vision [10], credit risk [8], spam detection [6], etc.”
Hybrid support vector machine rule extraction method for discovering the preferences of stock market investors: Evidence from Montenegro (2015, methodology): “By comparing the obtained results with the results obtained by Martens et al. (2007), on five different data sets, we have determined, same as they, that the rules extracted from SVM with help of DT have the accuracy which is comparable or better than the accuracy obtained with the DT method.”
A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations (2017, methodology): “For this purpose, we leverage a method developed by Martens and Provost [23], which computes, for a given instance which features need to be “removed” in order to change the classification outcome.”
Supervised classification and mathematical optimization (2013, background): “As pointed out in [170], in credit scoring “when credit has been denied to a customer, the Equal Credit Opportunity Act of the US requires that the financial institution provides specific reasons why the application was rejected; indefinite and vague reasons for denial are illegal.”
A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems (2010, methodology): “[180] Credit scoring Rule extraction from SVM C 4.”