Black Box Explanation by Learning Image Exemplars in the Latent Feature Space

By Riccardo Guidotti, Anna Monreale, Stan Matwin, Dino Pedreschi. 2019

77 citations recorded by the source corpus.

We present an approach to explain the decisions of black box models for image classification. While using the black box to label images, our explanation method exploits the latent feature space learned through an adversarial autoencoder. The proposed method first generates exemplar images in the latent feature space and learns a decision tree classifier. Then, it selects and decodes exemplars respecting local decision rules. Finally, it visualizes them in a manner that shows to the user how the exemplars can be modified to either stay within their class, or to become counter-factuals by "morphing" into another class. Since we focus on black box decision systems for image classification, the explanation obtained from the exemplars also provides a saliency map highlighting the areas of the image that contribute to its classification, and areas of the image that push it into another class. We present the results of an experimental evaluation on three datasets and two black box models. Besides providing the most useful and interpretable explanations, we show that the proposed method outperforms existing explainers in terms of fidelity, relevance, coherence, and stability.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Prototype Trees for Interpretable Fine-grained Image Recognition (2020, neutral): “activation maximization [39, 41] to visualize neurons, deconvolution or upconvolution [12, 54] to explain layers, generating image exemplars [18] to explain the latent space, or concept activation vectors [26] to explain model sensitivity.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “Another method named ABELE [69] generates exemplar and counter-exemplar images, labeled with the class identical to, and different from, the class of the image to explain, with a saliency map, highlighting the importance of the areas of the image contributing to its classification.”
Benchmarking and survey of explanation methods for black box models (2021, methodology): “Another crucial distinction highlighted in (Martens et al. 2007; Guidotti et al. 2019c; Adadi and Berrada 2018; Došilović et al. 2018; Carvalho et al. 2019) is between model-specific and modelagnostic explanation methods.”
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics (2021, background): “Example-based explanations provide a summarised overview of a model using representative examples or high-level concepts [80,90,91].”