Learning Credible Models

By Jiaxuan Wang, Jeeheh Oh, Jenna Wiens. 2017

30 citations recorded by the source corpus.

In many settings, it is important that a model be capable of providing reasons for its predictions (ıe, the model must be interpretable). However, the model's reasoning may not conform with well-established knowledge. In such cases, while interpretable, the model lacks credibility. In this work, we formally define credibility in the linear setting and focus on techniques for learning models that are both accurate and credible. In particular, we propose a regularization penalty, expert yielded estimates (EYE), that incorporates expert knowledge about well-known relationships among covariates and the outcome of interest. We give both theoretical and empirical results comparing our proposed method to several other regularization techniques. Across a range of settings, experiments on both synthetic and real data show that models learned using the EYE penalty are significantly more credible than those learned using other penalties. Applied to two large-scale patient risk stratification task, our proposed technique results in a model whose top features overlap significantly with known clinical risk factors, while still achieving good predictive performance.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Cynthia Rudin
2. Lesia Semenova
3. Chudi Zhong
4. Zhi Chen
5. Haiyang Huang
6. Berk Ustun
7. Xia Hu
8. Fan Yang

In Safety, Robustness & Interpretability

1. Lesia Semenova
2. Cynthia Rudin
3. Berk Ustun
4. Zhi Chen
5. Chudi Zhong
6. Haiyang Huang
7. Xia Hu
8. Mengnan Du

Citations in the corpus

• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “If we knew domain experts’ views about the importance of features, we should be able to incorporate that through regularization (Wang et al., 2018).”
• Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2018, background): “There is a spectrum between fully transparent models (where we understand how all the variables are jointly related to each other) and models that are lightly constrained in model form (such as models that are forced to increase as one of the variables increases, or models that, all else being equal”
• Please Stop Explaining Black Box Models for High Stakes Decisions (2018, background): “…are jointly related to each other) and models that are lightly constrained in model form (such as models that are forced to increase as one of the variables increases, or models that, all else being equal, prefer variables that domain experts have identified as important, see [Wang et al., 2018b]).”
• Learning Optimized Risk Scores (2016, background): “, 2016), credibility (Wang et al., 2018), and fairness (Kamishima et al.”
• Learning Credible Deep Neural Networks with Rationale Regularization (2019, methodology): “Specifically, We use the symmetric KL divergence between the normalized absolute value of explanation sn and the normalized rationale rn: symKL(sn, r ′ n) = 1 2 [KL(s ′ n||r n)+KL(r n||sn)] where lower divergence means higher credibility [5].”