From Local Explanations to Global Understanding with Explainable AI for Trees

By Scott M. Lundberg, Gabriel Erion-Barner, Hugh Chen, Alex J. DeGrave, J. Prutkin, Bala G. Nair, R. Katz, Jonathan Himmelfarb, Nisha Bansal, Su-In Lee. 2020

8,322 citations recorded by the source corpus.

Tree-based machine learning models such as random forests, decision trees and gradient boosted trees are popular nonlinear predictive models, yet comparatively little attention has been paid to explaining their predictions. Here we improve the interpretability of tree-based models through three main contributions. (1) A polynomial time algorithm to compute optimal explanations based on game theory. (2) A new type of explanation that directly measures local feature interaction effects. (3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to (1) identify high-magnitude but low-frequency nonlinear mortality risk factors in the US population, (2) highlight distinct population subgroups with shared risk characteristics, (3) identify nonlinear interaction effects among risk factors for chronic kidney disease and (4) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model’s performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains. Tree-based machine learning models are widely used in domains such as healthcare, finance and public services. The authors present an explanation method for trees that enables the computation of optimal local explanations for individual predictions, and demonstrate their method on three medical datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, background): “…number of shortcomings to this approach around, for instance, the generation of data or the loss of accuracy, which have been addressed with enhancements (Botari et al. 2020; Visani et al. 2020; Zhang et al. 2019), or with alternative approaches (Lundberg et al. 2020, 2018; Lundberg and Lee 2017).”
On scientific understanding with artificial intelligence (2022, background): “[86] Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz,”
An open-source, high-performance tool for automated sleep staging (2021, methodology): “Features importance was measured on the full training set using the Shapley Additive Explanation algorithm (SHAP; Lundberg et al., 2020 ).”
Explainable AI Methods - A Brief Overview (2020, methodology): “However, model-speciﬁc implementations exist for tree-based methods (random forest, xgboost etc.) or additive models [49].”
Practical guide to SHAP analysis: Explaining supervised machine learning model predictions in drug development (2024, positive): “26,28 To illustrate the use of these plots, we first trained an XGBoost model for the blood pressure regression problem introduced in the tutorial datasets.”
Explanations in Autonomous Driving: A Survey (2021, background): “We adapt terminologies from the explainable AI (XAI) domain.”
Algorithmic Bias in Education (2021, background): “Modern explainable AI methods make it possible to see the impact of a specific variable on prediction (Lundberg et al., 2020), making it easier to see not just where the biases are, but what variables might be causing the biases, towards developing better, less-biased algorithms.”
Improving performance of deep learning models with axiomatic attribution priors and expected gradients (2020, methodology): “Unlike previous methods which take additional steps to smooth saliency maps after training [43, 9], these are unmodified saliency maps directly from the learned model.”
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications (2021, methodology): “sex of the participant (figure is adapted from [115]).”
Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond (2021, methodology): “Suh et al. [75] developed a risk calculator model for prostate cancer (PCa) and clinically significant PCa with XAI modules that used Shapley value to determine the feature importance [143].”
Explainable Machine Learning in Credit Risk Management (2020, positive): “In particular, we use TreeSHAP method (Lundberg et al. 2020) [see e.g. Murdoch et al. (2019); Molnar 2019)] in combination with XGBoost.”
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI (2021, methodology): “TA1 and TA2 wished for an improved sorting feature that allows sorting by SHAP values to group observations with similar effects close to each other to identify regularities.”