Explaining prediction models and individual predictions with feature contributions

By Erik Strumbelj, Igor Kononenko. 2014

2,151 citations recorded by the source corpus.

Subfields: Artificial Intelligence, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In AI Ethics, Fairness & Society

Citations in the corpus

Consistent Individualized Feature Attribution for Tree Ensembles (2018, methodology): “SHAP values are theoretically optimal, but like other model agnostic feature attribution methods [2, 9, 21, 27], they can be challenging to compute.”
Explainable machine learning in deployment (2019, background): “These explanation functionals roughly fall into two categories: perturbation-based techniques [9, 16, 23, 35, 48, 58] and gradient-based techniques [8, 41, 53, 55, 56, 59].”
Learning Credible Models (2017, background): “…and l 2 θ represents the model parameters; β ∈ [ 0 , 1 ] is a hyperparameter that controls the tradeoff between the l 1 and l 2 norms; w is a set of non-negative weights for each feature; w ∗ | θ | [ i ] is the i th largest parameter sorted by magnitude; and ⊙ is the training time, Sun et al .…”
An unexpected unity among methods for interpreting model predictions (2016, methodology): “The thread of unity that this representation weaves through the literature indicates that there are common principles to be learned about the interpretation of model predictions that apply in many scenarios.”
Learning Explainable Models Using Attribution Priors (2019, background): “Previous literature about such methods has focused on the axioms they should satisfy [18, 37, 35, 6], and how attribution methods can give us insight into model behavior [19, 20, 29, 41].”
iml: An R package for Interpretable Machine Learning (2018, background): “• Shapley value (Strumbelj et al. 2014): Explaining single predictions by fairly distributing the predicted value among the features.”
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, background): “values [76], all combinations of variables must be considered when computing variable contributions.”
Algorithms to estimate Shapley value feature attributions (2022, background): “Then, LinearSHAP [15, 68] gives the following result for the marginal or baseline Shapley values:”
On Learning Sparse Boolean Formulae for Explaining AI Decisions (2017, background): “Domainindependent approaches have also been proposed in the AI literature for detecting sensitive input components that determine the decision in a classification problem [29,34].”
Explainable artificial intelligence: a comprehensive review (2021, methodology): “The idea of interpreting the AI system was first introduced in the mid-90s when Swartout and Moore (1993) proposed an XAI prototype that could describe what hard-coded rules contributed to a decision. The XAI phrase was initially suggested by Van Lent et al. (2004) in order to define the system capa”
Alignment for Advanced Machine Learning Systems (2020, background): “In a related vein, Datta, Sen, and Zick (2016), ˇStrumbelj and Kononenko (2014), and Robnik-ˇSikonja and Kononenko (2008) deﬁne metrics for reporting the inﬂuence of various inputs and sets of inputs on the output of a black-box ML system.”
The many Shapley values for model explanation (2019, methodology): “For instance, [10], assumes an independent feature distribution while [2] applies it to modules of a deep network rather than end-to-end.”
An Introduction to Machine Learning (2020, methodology): “6,7 Today, many aspects of a black box model can be interpreted using proper tools.”