InterpretML: A Unified Framework for Machine Learning Interpretability

By Harsha Nori, Samuel Jenkins, Paul Koch, R. Caruana. 2019

655 citations recorded by the source corpus.

InterpretML is an open-source Python package which exposes machine learning interpretability algorithms to practitioners and researchers. InterpretML exposes two types of interpretability - glassbox models, which are machine learning models designed for interpretability (ex: linear models, rule lists, generalized additive models), and blackbox explainability techniques for explaining existing systems (ex: Partial Dependence, LIME). The package enables practitioners to easily compare interpretability algorithms by exposing multiple methods under a unified API, and by having a built-in, extensible visualization platform. InterpretML also includes the first implementation of the Explainable Boosting Machine, a powerful, interpretable, glassbox model that can be as accurate as many blackbox models. The MIT licensed source code can be downloaded from github.com/microsoft/interpret.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “Figure 5: Left: All component functions of a GAM model trained using the interpret package (Nori et al., 2019) on a diabetes dataset (Dua and Graff, 2017); Right: zoom-in of component function for glucose concentration.”
Neural Additive Models: Interpretable Machine Learning with Neural Nets (2020, methodology): “We use the open-source implementation [Nori et al., 2019] with the parameters specified by prior work [Caruana et al.”
Manipulating and Measuring Model Interpretability (2018, methodology): “[49] studied data scientists’ use of two specific interpretability tools (the InterpretML [81] implementation of generalized additive models and the SHAP Python package), finding that data scientists over-trust and misuse these tools.”
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models (2020, background): “Many are general-purpose, such as the What-If Tool (Wexler et al., 2020), Cap-tum (Kokhlikyan et al., 2019), Manifold (Zhang et al., 2018), or InterpretML (Nori et al., 2019), while others focus on speciﬁc applications like fairness, including FairVis (Cabrera et al., 2019) and FairSight (Ahn and Li”
Robustness Gym: Unifying the NLP Evaluation Landscape (2021, neutral): “, 2019b), InterpretML (Nori et al., 2019), Manifold (Zhang et al.”
Capabilities of GPT-4 on Medical Challenge Problems (2023, background): “In summary, with the addition of recent NSAID use, acute kidney injury becomes the most likely explanation for the patient’s symptoms and lab ﬁndings.”
Captum: A unified and generic model interpretability library for PyTorch (2020, methodology): “Existing frameworks such as DeepExplain [3], Alibi [8] and Inter-pretML [11] have been developed to unify those algorithms in one framework and make them accessible to all machine learning model developers and practitioners.”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, methodology): “Open source software with implementations of various IML methods is available, for example, iml [76] and DALEX [11] for R [91] and Alibi [58] and InterpretML [83] for Python.”
Ethical Considerations in AI-Based Recruitment (2019, methodology): “ InterpretML [37]: Though currently in alpha, this project has high engagement on GitHub.”
Automated assessment of psychiatric disorders using speech: A systematic review (2019, background): “For a practical perspective, there is a book covering many useful tools(135) and packages that compare multiple explainability methods.(136) Explaining which features are predictive of”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “The beginner-friendly Microsoft toolbox InterpretML [100] from 2019 implements five model-agnostic and four transparent XAI methods that are shortly introduced in the paper.”
A Survey of Deep Learning for Scientific Discovery (2020, methodology): “These methods can typically be less inter-pretable than linear/logistic regression, but recent work [160] has looked at developing software libraries https://github . com/interpretml/interpret to address this challenge.”
Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI (2020, methodology): “Researchers have accompanied the proliferation of AI ethics principles by creating mathematical methods and software toolkits for developing fairer [8,75,76], more inter-pretable [72], and privacy-preserving AI systems [45].”
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations (2020, methodology): “They have been used in healthcare [9], to explain black-box models [61], in explanation interfaces for empirical studies [4, 26] and included in recent toolkits to strengthen the support for using GAMs [44, 54].”
Interpreting Interpretability: Understanding Data Scientists' Use of Interpretability Tools for Machine Learning (2020, background): “InterpretML also provides global explanations (see Figure 1, top left) and local explanations (see Figure 1, top right) by ranking and sorting the contributions made by each input feature to the predictions [50].”