Explainable AI for Trees: From Local Explanations to Global Understanding

By Scott M. Lundberg, Gabriel Erion-Barner, Hugh Chen, Alex J. DeGrave, J. Prutkin, Bala G. Nair, R. Katz, Jonathan Himmelfarb, Nisha Bansal, Su-In Lee. 2019

385 citations recorded by the source corpus.

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the general US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Explaining Models by Propagating Shapley Values of Local Components (2019, methodology): “The dataset has 79 features for 14,407 individuals released by (Lundberg et al. 2018) In Figure 3, we plot a summary of DeepSHAP (with 1000 random background samples) attributions for all NHANES training samples (n= 8023 ) and notice a few trends.”
Learning Explainable Models Using Attribution Priors (2019, methodology): “The optimal regularization range was found to be λ ∈ [10, 10].”
Analyzing ImageNet with Spectral Relevance Analysis: Towards ImageNet un-Hans'ed (2019, methodology): “XAI methods can be roughly categorized into two groups: methods providing local explanations and those providing global explanations [9].”
Understanding the diversity of the metal-organic framework ecosystem (2020, methodology): “The second and third approach are permutation importance and SHapley Additive exPlanations (SHAP), 48 respectively, which were computed for the test or train set.”
Explaining Patterns in Data with Language Models via Interpretable Autoprompting (2022, methodology): “…interpretation A popular method for interpreting neural networks is to inspect an LLM’s individual predictions via feature importances [29, 30], feature-interaction importances [31, 32], extractive rationales [33, 34], or natural-language explanations for individual predictions [35,…”
Shapley values for feature selection: The good, the bad, and the axioms (2021, positive): “In Sections IV-A and IV-B, we reproduce the results of [53, Theorem 8 and 9] and extend them via simulation to include the SHAP FSelection formulation [48] (i.e., feature selection by ranking mean absolute SHAP values of model predictions), and the SAGE formulation.”
Epistemic values in feature importance methods: Lessons from feminist epistemology (2021, background): “Test subjects are often recruited via general surveying sites like Amazon’s Mechanical Turk, and asked to evaluate explanations with little to no context about why the predictions are being made and what purposes the explanations would serve in a real-world applications [49, 61, 66, 69].”