Do Not Trust Additive Explanations

By Alicja Gosiewska, Przemyslaw Biecek. 2019

58 citations recorded by the source corpus.

Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. But how faithful are these additive explanations? Can we rely on additive explanations for non-additive models? In this paper, we (1) examine the behavior of the most popular instance-level explanations under the presence of interactions, (2) introduce a new method that detects interactions for instance-level explanations, (3) perform a large scale benchmark to see how frequently additive explanations may be misleading.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “, 2020) as well as uncertainty bands for variable importance (Gosiewska and Biecek, 2020; Fisher et al., 2019).”
Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models (2020, background): “Analogous ﬁndings also in Gosiewska and Biecek (2019).”
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI (2021, methodology): “(ii) Label/mechanism confusion through subtle interactions: Shapley explanations hide much of the model’s complex behavior behind accessible feature value attributions [20].”