CXPlain: Causal Explanations for Model Interpretation under Uncertainty

By Patrick Schwab, Walter Karlen. 2019

247 citations recorded by the source corpus.

Feature importance estimates that inform users about the degree to which given inputs influence the output of a predictive model are crucial for understanding, validating, and interpreting machine-learning models. However, providing fast and accurate estimates of feature importance for high-dimensional data, and quantifying the uncertainty of such estimates remain open challenges. Here, we frame the task of providing explanations for the decisions of machine-learning models as a causal learning task, and train causal explanation (CXPlain) models that learn to estimate to what degree certain inputs cause outputs in another machine-learning model. CXPlain can, once trained, be used to explain the target model in little time, and enables the quantification of the uncertainty associated with its feature importance estimates via bootstrap ensembling. We present experiments that demonstrate that CXPlain is significantly more accurate and faster than existing model-agnostic methods for estimating feature importance. In addition, we confirm that the uncertainty estimates provided by CXPlain ensembles are strongly correlated with their ability to accurately estimate feature importance on held-out data.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning (2021, background): “Representative works include SHAP (Lundberg & Lee, 2017) and its variants (Lundberg et al., 2018; Shrikumar et al., 2017; Schwab & Karlen, 2019), which assign contributions to each feature used in a model, and the sum of the contributions is the difference between a model’s output and the target…”
XOC: Explainable Observer-Classifier for Explainable Binary Decisions (2019, background): “d a follow-up work [42] uses a classical decision tree to explain the predictions based on the learned object-part ﬁlters. [5] establishes the connection between causality and network attribution and [33] propose a model agnostic way of training a causal explanation model that interprets a given pre”
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2021, background): “For instance, interpreting model outputs and its con-ﬁdence is critical in autonomous vehicle driving, where wrong predictions are costly or harmful (Schwab and Karlen, 2019).”
Generative causal explanations of black-box classifiers (2020, methodology): “) Prior work has quantified the impact of features in data space by using Granger causality [13], a priori known causal structure [46, 36], an average or individual causal effect metric [47, 19], or by applying random valuedinterventions [48].”
Explaining by Removing: A Unified Framework for Model Explanation (2020, methodology): “…to how Occlusion, PredDiff and CXPlain calculate local feature importance and how permutation tests and feature ablation (LOCO) summarize global feature importance (Breiman, 2001; Strobl et al., 2008; Zeiler and Fergus, 2014; Zintgraf et al., 2017; Lei et al., 2018; Schwab and Karlen, 2019).”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, unknown): “[312], Smith et al.”