An Interpretable Model with Globally Consistent Explanations for Credit Risk

By Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, Tong Wang. 2018

104 citations recorded by the source corpus.

We propose a possible solution to a public challenge posed by the Fair Isaac Corporation (FICO), which is to provide an explainable model for credit risk assessment. Rather than present a black box model and explain it afterwards, we provide a globally interpretable model that is as accurate as other neural networks. Our "two-layer additive risk model" is decomposable into subscales, where each node in the second layer represents a meaningful subscale, and all of the nonlinearities are transparent. We provide three types of explanations that are simpler than, but consistent with, the global model. One of these explanation methods involves solving a minimum set cover problem to find high-support globally-consistent explanations. We present a new online visualization tool to allow users to explore the global model and its explanations.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Cynthia Rudin
2. Ana Marasovic
3. Tong Wang
4. Lesia Semenova
5. Yoav Goldberg
6. Berk Ustun
7. Chudi Zhong
8. Zhi Chen

In Safety, Robustness & Interpretability

1. Ana Marasovic
2. Cynthia Rudin
3. Lesia Semenova
4. Tong Wang
5. Zhi Chen
6. Chudi Zhong
7. Haiyang Huang
8. Berk Ustun

Citations in the corpus

• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “If we use an unsupervised algorithm for disentangling the space, and then evaluate whether it indeed disentangled the known concepts, then we have a useful quantitative evaluation (Higgins et al., 2017; Chen et al., 2018b; Kim and Mnih, 2018; Eastwood and Williams, 2018; Do and Tran, 2019).”
• Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2018, background): “ict credit default and explain the model afterwards. However, there was no performance difference between interpretable models and explainable models for the FICO data. A globally interpretable model [22] won the FICO Recognition Prize for the competition. This is a case where the organizers and jud”
• InterpretML: A Unified Framework for Machine Learning Interpretability (2019, background): “This can easily be observed in high-risk applications such as healthcare (Ahmad et al., 2018; Caruana et al., 2015), finance (Hajek, 2019; Chen et al., 2018) and judicial environments (Tan et al., 2018; Soundarajan and Clausen).”
• Learning Optimized Risk Scores (2016, background): “…promising option to build models that obey constraints related to, for example, interpretability (see e.g. Caruana et al., 2015; Gupta et al., 2016; Chen et al., 2018; Li et al., 2018, where interpretability is addressed through constraints on model form), safety (Amodei et al., 2016), credibility…”
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “For example, AI-produced credit scoring [9] represents a risk to the loan officer: a wrong decision carries a risk (among others) that the applicant defaults in the future.”
• Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “Decisions that can result inmultiplicity include choosingwhat information shouldbe includedas input to themodel [26], which points are included in the training set [10], which model class should be adopted [15], what random numbers the model’s parameters are initialized with [10, 41], among many oth”
• Model-Agnostic Linear Competitors - When Interpretable Models Compete and Collaborate with Black-Box Models (2019, background): “The interaction of models and humans naturally calls for users’ understanding of machine learning models, especially in high-stake applications such as healthcare, judiciaries, etc (Letham et al. 2015, Yang et al. 2018, Caruana et al. 2015, Chen et al. 2018).”
• Understanding Uncertainty in Bayesian Neural Networks (2019, background): “Alternatively, local explanations can be grouped into summaries or trends (Chen et al., 2018b).”
• Explainable AI under contract and tort law: legal incentives and technical challenges (2020, background): “The trade-off between performance and accuracy might only hold for a subset of particularly complex problems (Rudin 2019; Chen et al. 2018), such as image classification and machine translation.”
• Globally-Consistent Rule-Based Summary-Explanations for Machine Learning Models: Application to Credit-Risk Evaluation (2019, unknown): “We note in passing that a considerable effort in which we tried various transformations of the features and other discretization methods did not lead to improved predictions (Chen et al. 2018).”
• A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning (2019, background): “[12] Chaofan Chen, Kancheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang.”