Explaining Models by Propagating Shapley Values of Local Components
By Hugh Chen, Scott M. Lundberg, Su-In Lee. 2019
139 citations recorded by the source corpus.
In healthcare, making the best possible predictions with complex models (e.g., neural networks, ensembles/stacks of different models) can impact patient welfare. In order to make these complex models explainable, we present DeepSHAP for mixed model types, a framework for layer wise propagation of Shapley values that builds upon DeepLIFT (an existing approach for explaining neural networks). We show that in addition to being able to explain neural networks, this new framework naturally enables attributions for stacks of mixed models (e.g., neural network feature extractor into a tree model) as well as attributions of the loss. Finally, we theoretically justify a method for obtaining attributions with respect to a background distribution (under a Shapley value framework).
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Eduardo Almeida Soares
- Plamen Parvanov Angelov
- Amitojdeep Singh
- Jun Xia
- Guang Yang
- Qinghao Ye
- Vasudevan Lakshminarayanan
- Sourya Sengupta
In Safety, Robustness & Interpretability
- Eduardo Almeida Soares
- Plamen Parvanov Angelov
- Amitojdeep Singh
- Jun Xia
- Guang Yang
- Qinghao Ye
- Vasudevan Lakshminarayanan
- Sourya Sengupta
Citations in the corpus
- Explainable artificial intelligence: an analytical review (2021, methodology): “Roughly, this type of XAI approaches attempts to evaluate a feature based on its average expected marginal contribution to the model's decision, after all possible combinations have been considered (Chen et al., 2019; Pedreschi et al., 2019).”
- Explainable Deep Learning Models in Medical Image Analysis (2020, methodology): “Deep SHAP [30] It is a fast approximation algorithm to compute the game theory based SHAP values .”
- Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond (2021, methodology): “A recent comparative study of different models of XAI, including DeepLIFT [145], Guided backpropagation (GBP) [146], Layer wise relevance propagation (LRP) [147], SHapley Additive exPlanations (SHAP) [148] and others, was conducted for ophthalmic diagnosis [76].”