Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

By Cynthia Rudin, Zhi Chen, Haiyang Huang, Lesia Semenova, Chudi Zhong. Statistics Survey, 2021

986 citations recorded by the source corpus.

Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the"Rashomon set"of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

View Paper: https://arxiv.org/abs/2103.11251

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Models constrained to be more understandable to people are easier to troubleshoot and more useful in practice (Rudin et al., 2021).”
Ethical Implementation of Artificial Intelligence to Select Embryos in In Vitro Fertilization (2021, neutral): “An interpretable ML model is a predictive model that is constrained so that a human can better understand its reasoning process [26, 59].”
Neural Additive Models: Interpretable Machine Learning with Neural Nets (2020, background): “While NAMs only use some of the expressivity of DNNs, one can imagine using NAMs in a realworld pipeline where intelligibility is required for decision making from representation [36] (e.”
SurvNAM: The machine learning survival model explanation (2021, background): “Descriptions of many explanation methods and various approaches, their critical reviews can be found in survey papers [46, 47, 48, 4, 49, 50].”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, background): “The interoperability of PINN will also play an important role in future research (Rudin et al, 2022).”
Visual Classification via Description from Large Language Models (2022, methodology): “These text embeddings are similar to class prototypes (Snell et al., 2017; Rudin et al., 2021), but instead they are obtained across modalities, as discussed further in the Related Work.”
Globally-Consistent Rule-Based Summary-Explanations for Machine Learning Models: Application to Credit-Risk Evaluation (2019, neutral): “…focus on this research area and refer the reader to survey papers on interpretable and explainable machine learning to other bodies of work (Rudin et al. 2022, Došilović et al. 2018, Guidotti et al. 2018b, Arya et al. 2020, Tjoa and Guan 2020, Roscher et al. 2020, Burkart and Huber 2021).”