Explaining Image Classifiers by Counterfactual Generation

By Chun-Hao Kingsley Chang, Elliot Creager, Anna Goldenberg, David Kristjanson Duvenaud. 2018

298 citations recorded by the source corpus.

When an image classifier makes a prediction, which parts of the image are relevant and why? We can rephrase this question to ask: which parts of the image, if they were not seen by the classifier, would most change its decision? Producing an answer requires marginalizing over images that could have been seen but weren't. We can sample plausible image in-fills by conditioning a generative model on the rest of the image. We then optimize to find the image regions that most change the classifier's decision after in-fill. Our approach contrasts with ad-hoc in-filling approaches, such as blurring or injecting noise, which generate inputs far from the data distribution, and ignore informative relationships between different parts of the image. Our method produces more compact and relevant saliency maps, with fewer artifacts compared to previous methods.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks (2019, methodology): “The resulting explanations (Fig 1 b) are valid model inputs and their faithfulness can, thus, be directly verified (as in methods from [17, 14, 6, 11]).”
Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, methodology): “As a second baseline, we adapt FIDO (Chang et al., 2019), a counterfactual feature importance method, to explain uncertainty.”
Explaining Classifiers with Causal Concept Effect (CaCE) (2019, methodology): “Many interpretability methods developed to have causalflavor are for local explanations, such as removing and adding pixels to generate counterfactual explanations for images (Goyal et al., 2019; Chang et al., 2018) or for texts (Hendricks et al.”
Counterfactual Visual Explanations (2019, methodology): “In similar spirit, a concurrent work (Chang et al., 2018) uses a trained generative model to fill in the masked image regions conditioned on the unmasked regions of the image.”
Generative causal explanations of black-box classifiers (2020, methodology): “Generative models have been used to compute interventions that respect the data distribution [51, 36, 19, 52], a key idea in this paper.”
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks (2019, background): “Perturbation-Based approaches [3,5,6,9,10,19] work by perturbing original input and observe the change of the prediction of model.”
Aggregating explanation methods for stable and robust explainability. (2019, result): “The open problem of explainability is reflected in a lot of recent work (Kindermans et al., 2017; Selvaraju et al., 2017; Bach et al., 2015; Zhang et al., 2018a; Zhou et al., 2016; Ancona et al., 2018; Ribeiro et al., 2016; Rieger et al., 2018; Kim et al., 2018; Lundberg & Lee, 2017; Zintgraf et al.”
Explaining by Removing: A Unified Framework for Model Explanation (2020, methodology): “• (Generative model) FIDO-CA (Chang et al., 2018) removes features by replacing them with samples from a conditional generative model (e.”
On the (In)fidelity and Sensitivity for Explanations. (2019, methodology): “This was further improved by [49, 7] by including a generative model, similar in spirit to counterfactual visual explanations [15].”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “We hope the taxonomy can shed light on future improvements/extensions on explaining Table 2 List of interpretation algorithm publications Methods Publications (non-exhaustive) LIME and variants LIME [137], Anchors [138], SHAP [110], RISE [127], MAPLE [130] Global interpretation LIME-SP [137], NormLI”
Distilling Model Failures as Directions in Latent Space (2022, unknown): “…et al., 2013; Dabkowski & Gal, 2017; Sundararajan et al., 2017) and perturbation-based (Ribeiro et al., 2016a; Goyal et al., 2019; Fong & Vedaldi, 2017; Dabkowski & Gal, 2017; Zintgraf et al., 2017; Dhurandhar et al., 2018; Chang et al., 2019; Hendricks et al., 2018; Singla et al., 2021) methods.”
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models (2020, background): “In this study, we focus on post hoc saliency mapping strategies, where saliency maps are constructed given an arbitrary (trained) prediction model, as opposed to relying on customized model architectures for interpretable predictions [19, 61], or to train a separate module to explicitly produce mode”
Understanding Uncertainty in Bayesian Neural Networks (2019, methodology): “ARD Automatic Relevance Determination BNN Bayesian Neural Network CLUE Counterfactual Latent Uncertainty Explanation DGM Deep Generative Model DLV M Deep Latent Variable Model ELBO Evidence Lower Bound EP Expectation Propagation FGSM Fast Gradient Sign Method FIDO Fill-In the DropOut region GDPR Gen”
Evaluating and Aggregating Feature-based Model Explanations (2020, background): “[Chang et al., 2019] notes that the baseline can be learned using generative modeling.”
Explaining Visual Models by Causal Attribution (2019, background): “[3] also use pixel masking, not only by replacing with a constant value, random noise or by blurring the masked region, but by infilling it with probable values with respect to the remaining parts of the image.”
Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias (2019, background): “Another set of approaches determine importance based on perturbing individual or groups of pixels and observing the change in classifier output [6, 10, 8].”
Towards causal benchmarking of bias in face analysis algorithms (2020, background): “One strategy is to determine regions of the input that are salient, either through analysis of gradients or perturbations of the input image [13, 16, 24, 26, 69, 72, 74].”