Explaining Classifiers with Causal Concept Effect (CaCE)

By Yash Goyal, Amir Feder, Uri Shalit, Been Kim. 2019

213 citations recorded by the source corpus.

How can we understand classification decisions made by deep neural networks? Many existing explainability methods rely solely on correlations and fail to account for confounding, which may result in potentially misleading explanations. To overcome this problem, we define the Causal Concept Effect (CaCE) as the causal effect of (the presence or absence of) a human-interpretable concept on a deep neural net's predictions. We show that the CaCE measure can avoid errors stemming from confounding. Estimating CaCE is difficult in situations where we cannot easily simulate the do-operator. To mitigate this problem, we use a generative model, specifically a Variational AutoEncoder (VAE), to measure VAE-CaCE. In an extensive experimental analysis, we show that the VAE-CaCE is able to estimate the true concept causal effect, compared to baselines for a number of datasets including high dimensional images.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Causal Inference & Discovery

Citations in the corpus

CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, methodology): “Using generative models has been recently discussed in the case of images (Goyal et al. 2019a).”
Concept Bottleneck Models (2020, background): “See, e.g., Goyal et al. (2019); O’Shaughnessy et al. (2020) for discussions on causality in the context of interpreting models with concepts.”
Post-hoc Interpretability for Neural NLP: A Survey (2021, background): “The term concept is much more common in computer vision [44, 59, 80] than in NLP.”
Benchmarking and survey of explanation methods for black box models (2021, background): “CaCE, Causal Concept Effect (Goyal et al. 2019) is another variation of tcav that looks at the causal effect of the presence or absence of high-level concepts on the black-box prediction.”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, methodology): “[42] measures the causal effect of concepts by using a conditional VAE model.”
Generative causal explanations of black-box classifiers (2020, background): “compute the average causal effect (ACE) of human-interpretable concepts on the classifier [50], but require labeled examples of the concepts and suffer from limitations of the ACE metric [8].”
Post-hoc Concept Bottleneck Models (2022, background): “Estimating the causal effect of a concept requires careful treatment (Goyal et al., 2019), e.g. accounting for the interactions between concepts, and we leave this to future work.”
Counterfactual Generative Networks (2021, background): “Most previous work (Goyal et al., 2019; Suter et al., 2019) imposes no restrictions on f , i.”
Explaining in Style: Training a GAN to explain a classifier in StyleSpace (2021, methodology): “Other explanation methods generate counterfactuals using attributes, where full or partial supervision for the desired attributes is available [10, 5].”
Acquisition of chess knowledge in AlphaZero (2021, background): “While there exist causal concept-based methods (30, 31), this work focuses on analysis without having to assume a causal graph that could incorrectly represent the internal mechanism.”
What if This Modified That? Syntactic Interventions with Counterfactual Embeddings (2021, background): “Goyal et al. (2019) and Kaushik et al. (2020) generate counterfactual inputs to language models, while Vig et al. (2020) study individual neurons and attention heads to uncover gender biases in pre-trained networks.”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey (2020, methodology): “Newer methods proposed in literature such as explaining with Concepts [51, 52, 58] which we discussed in subsection”