Explaining individual predictions when features are dependent: More accurate approximations to Shapley values

By Kjersti Aas, Martin Jullum, Anders Løland. 2019

980 citations recorded by the source corpus.

Explaining complex or seemingly simple machine learning models is a practical and ethical question, as well as a legal issue. Can I trust the model? Is it biased? Can I explain it to others? We want to explain individual predictions from a complex machine learning model by learning simple, interpretable explanations. Of existing work on interpreting complex models, Shapley values is the only method with a solid theoretical foundation. Kernel SHAP is a computationally efficient approximation to Shapley values in higher dimensions. Like most other existing methods, this approach assumes independent features, which may give very wrong explanations. This is the case even if a simple linear model is used for predictions. We extend the Kernel SHAP method to handle dependent features. We provide several examples of linear and non-linear models with linear and non-linear feature dependence, where our method gives more accurate approximations to the true Shapley values. We also propose a method for aggregating individual Shapley values, such that the prediction can be explained by groups of dependent variables.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME (2023, methodology): “Although there are some works which tried to deal with the issue of collinearity [19, 20], yet the proposed methods are either limited to a local explanation [19] or the explanation is user-dependent [20].”
Deep neural network or dermatologist? (2019, methodology): “Finally, there exists recent, alternative methods for implementing Shapley analysis that may well produce better results and permit the use of higher resolution images [5,1] .”
The many Shapley values for model explanation (2019, methodology): “(6) [12] computes the Shapley value with conditional expectations efficiently for trees; however, it is not very clear about its assumptions on the feature distribution 1 (7) [13] generalizes one of the approaches in [2] to the case when the distributions are not independent, either by assuming that”
Explaining by Removing: A Unified Framework for Model Explanation (2020, background): “Similarly, Ancona et al. (2017) showed that Grad * Input, DeepLIFT, LRP and Integrated Gradients are all understandable as modified gradient backpropagations.”
Algorithms to estimate Shapley value feature attributions (2022, background): “This question is frequently debated in Shapley value literature, with some papers defending marginal Shapley values [23, 24], others advocating for conditional Shapley values [26, 33, 49], and still others arguing for causal solutions [25].”
Understanding Global Feature Contributions With Additive Importance Measures (2020, background): “This approach is common in recent work [1, 25] and is necessary for later connections with mutual information (Supplement A).”
Feature relevance quantification in explainable AI: A causality problem (2019, methodology): “We take the kernel estimation method from Aas et al. (2019) to show how strongly the Shapley Values w.”
On the Tractability of SHAP Explanations (2020, background): “More recently there has been a lot of work on the tradeoffs of variants of the original S HAP explanations, e.g., Sundararajan and Najmi (2020), Kumar et al. (2020), Janzing, Minorics, and Bloebaum (2020), Merrick and Taly (2020), and Aas, Jullum, and Løland (2019).”