Interpretable Basis Decomposition for Visual Explanation

By Bolei Zhou, Yiyou Sun, David Bau, Antonio Torralba. 2018

328 citations recorded by the source corpus.

Explanations of the decisions made by a deep neural network are important for human end-users to be able to understand and diagnose the trustworthiness of the system. Current neural networks used for visual recognition are generally used as black boxes that do not provide any human interpretable justification for a prediction. In this work we propose a new framework called Interpretable Basis Decomposition for providing visual explanations for classification networks. By decomposing the neural activations of the input image into semantically interpretable components pre-trained from a large concept corpus, the proposed framework is able to disentangle the evidence encoded in the activation feature vector, and quantify the contribution of each piece of evidence to the final prediction. We apply our framework for providing explanations to several popular networks for visual recognition, and show it is able to explain the predictions given by the networks in a human-interpretable way. The human interpretability of the visual explanations provided by our framework and other recent explanation methods is evaluated through Amazon Mechanical Turk, showing that our framework generates more faithful and interpretable explanations (The code and data are available at https://github.com/CSAILVision/IBD).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “Even if we create a vector in the latent space that is aimed towards a single concept (as is done in Kim et al., 2018; Zhou et al., 2018b), that vector could activate highly on multiple concepts, which means the signal for the two concepts is not disentangled.”
Learning to Explain With Complemental Examples (2018, background): “The primary stream is visualizing where the classifier weighs for its prediction by assigning an importance to each element in the input space, by propagating the prediction to the input space [24, 3, 29, 23, 30, 8, 31], or by learning the instance-wise importance of elements [4, 5, 15] with an auxi”
Interpreting CNNs via Decision Trees (2018, neutral): “[45] explained semantic reasons for visual recognition.”
Neural Prototype Trees for Interpretable Fine-grained Image Recognition (2020, methodology): “part-based methods [56, 61], saliency maps [6, 13, 60] or representer points [53].”
Concept Bottleneck Models (2020, methodology): “Non-bottleneck models that use human-specified concepts commonly use them in auxiliary objectives in a multi-task setup, or as auxiliary features; examples include using object parts (Huang et al., 2016; Zhou et al., 2018), parse trees (Zelenko et al.”
Concept whitening for interpretable image recognition (2020, methodology): “Efforts to interpret individual nodes of pre-trained neural networks (e.g., Zhou et al., 2018a; 2014) have shown that some fraction of nodes can be identiﬁed to be aligned with some high-level semantic meaning, but these special nodes do not provably contain the network’s full information about the…”
This looks like that: deep learning for interpretable image recognition (2018, background): “Ns. These models aim to expose the parts of an input the network focuses on when making decisions. Examples of attention models include class activation maps [56] and various part-based models (e.g., [55, 53, 15, 57, 43, 10, 9, 34, 37, 49, 7]; see Table 1). However, attention-based models can only t”
Explaining machine learning classifiers through diverse counterfactual explanations (2019, background): “Such visualizations are commonly used in the computer vision community, ranging from highlighting certain parts of an image to activations in convolutional neural networks [24, 34, 35].”
Label-Free Concept Bottleneck Models (2023, methodology): “…residual fitting layers is often significantly lower than the standard DNNs. Similarly, an earlier work Interpretable Basis Decomposition (Zhou et al., 2018) proposes learning a concept bottleneck layer based on labeled concept data for explanable decisions, even though they do not call…”
Towards Automatic Concept-based Explanations (2019, background): “Following the literature [46, 20], throughout this work, we refer to these units as concepts.”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “Researchers have proposed many attention methods [11], [12], [13], [14], [15] to locate the object regions of the class of interest from the powerful CNN based image classiﬁers [16], [17], [18], [19], [20], [21], [22].”
This Looks Like That: Deep Learning for Interpretable Image Recognition (2018, background): “[41, 13, 44]), or rely on an auxiliary (pre-trained) network to extract image patches for unsupervised identification of important parts (e.”
Trustworthy Artificial Intelligence: A Review (2022, background): “Other researchers [228] have proposed visualizing the contribution of the evidence for the final decision to provide explainability to classifiers.”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, methodology): “Our metric can be applied to a set of concept vectors that lie in a subspace of some intermediate DNN activations, which is a general assumption in previous work in this context [5, 6].”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, background): “Some recent work has also focused on creating conceptbased explanations [18, 59, 9].”
On Concept-Based Explanations in Deep Neural Networks (2019, background): “[32] decompose the prediction of a data sample into linear combinations of concept components.”
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems (2020, background): “In explainable AI literature, subjective measures, such as user trust and experience, have been largely embraced as the focal point for the evaluation of explainable systems [35, 36, 44, 48].”
Multimodal Explanations by Predicting Counterfactuality in Videos (2018, background): “In the latter category, the main goal is to know where the model actually “looks” in the input by propagating the prediction to the input space [24, 4, 32, 23, 33, 9, 34], or by learning instance-wise importance of elements [6, 8] with an auxiliary model.”