On Completeness-aware Concept-Based Explanations in Deep Neural Networks

By Chih-Kuan Yeh, Been Kim, Sercan Ö. Arik, Chun-Liang Li, Tomas Pfister, Pradeep Ravikumar. 2019

373 citations recorded by the source corpus.

Human explanations of high-level decisions are often expressed in terms of key concepts the decisions are based on. In this paper, we study such concept-based explainability for Deep Neural Networks (DNNs). First, we define the notion of completeness, which quantifies how sufficient a particular set of concepts is in explaining a model's prediction behavior based on the assumption that complete concept scores are sufficient statistics of the model prediction. Next, we propose a concept discovery method that aims to infer a complete set of concepts that are additionally encouraged to be interpretable, which addresses the limitations of existing methods on concept explanations. To define an importance score for each discovered concept, we adapt game-theoretic notions to aggregate over sets and propose ConceptSHAP. Via proposed metrics and user studies, on a synthetic dataset with apriori-known concept explanations, as well as on real-world image and language datasets, we validate the effectiveness of our method in finding concepts that are both complete in explaining the decisions and interpretable. (The code is released at this https URL)

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, methodology): “Following Jain et al. (2020), we use the same experimental setup and saliency-based base-lines such as attention (Lei et al., 2016; Bastings et al., 2019) and gradient (Li et al., 2016) based explanation methods.”
Concept whitening for interpretable image recognition (2020, background): “To address this weakness, several concept-based post-hoc explanation approaches have recently been proposed that do not rely on the concept aligning with individual units (Kim et al., 2017; Zhou et al., 2018b; Ghorbani et al., 2019; Yeh et al., 2019).”
Explaining in Style: Training a GAN to explain a classifier in StyleSpace (2021, methodology): “The first part uses a setup similar to [38].”
Distilling Model Failures as Directions in Latent Space (2022, neutral): “Other works analyze the model’s activations (Bau et al., 2017; Kim et al., 2018; Yeh et al., 2020; Wong et al., 2021).”
Post-hoc Concept Bottleneck Models (2022, background): “The concepts in the library can be selected by a domain expert or learned automatically from the data (Ghorbani et al., 2019; Yeh et al., 2020).”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “[144] additionally introduces the notion of completeness of a set of concepts as the maximum performance of the model intercepted by the semantic bottleneck.”