Deep Learning for Case-based Reasoning through Prototypes: A Neural Network that Explains its Predictions

By Oscar Li, Hao Liu, Cynthia Rudin. 2017

677 citations recorded by the source corpus.

Deep neural networks are widely used for classification. These deep models often suffer from a lack of interpretability---they are particularly difficult to understand because of their non-linear nature. As a result, neural networks are often treated as "black box" models, and in the past, have been trained purely to optimize the accuracy of predictions. In this work, we create a novel network architecture for deep learning that naturally explains its own reasoning for each prediction. This architecture contains an autoencoder and a special prototype layer, where each unit of that layer stores a weight vector that resembles an encoded training input. The encoder of the autoencoder allows us to do comparisons within the latent space, while the decoder allows us to visualize the learned prototypes. The training objective has four terms: an accuracy term, a term that encourages every prototype to be similar to at least one encoded input, a term that encourages every encoded input to be close to at least one prototype, and a term that encourages faithful reconstruction by the autoencoder. The distances computed in the prototype layer are used as part of the classification process. Since the prototypes are learned during training, the learned network naturally comes with explanations for each prediction, and the explanations are loyal to what the network actually computes.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “[129] took the concepts in [183] and [53], and integrated prototype learning into recurrent neural networks for modeling sequential”
EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction (2019, background): “Their work can be seen as a generalization of [16].”
The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning (2019, methodology): “Finally, most previous twin-system work has tended to examine traditional neural networks (i.e., MLPs), rather than more recent DNNs (for rare potential exceptions see [Chen et al., 2018] and [Li et al., 2017]).”
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2018, background): “In a recent attempt to do this, Chen, Li, and colleagues have been building architectures that append a special prototype layer to the end of the network [49, 55].”
Please Stop Explaining Black Box Models for High Stakes Decisions (2018, background): “In a recent attempt to do this, Li et al. [2018], Chen et al. [2018a] have been building architectures that append a special prototype layer to the end of the network.”
Learning Optimal Decision Trees with SAT (2018, background): “The importance of XAI is illustrated by a range of work in the recent past ([Li et al. , 2018; Lakkaraju et al. , 2017; Angelino et al. , 2017; Lakkaraju et al. , 2016; Letham et al. , 2015; Lou et al. , 2012] among others), by a number of recent events dedicated to the topic [IJ-CAI XAI Workshop,…”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “A by-product of the approach introduced in Section IV is that it returns exemplar inputs, also called prototypes [55], [56], to interpret predictions through training points that best explain the model’s output because they are processed similarly to the”
Learning Certifiably Optimal Rule Lists (2017, background): “al combined classier. The notions of interpretability used in image classication tend to be completely dierent from those for structured data where each feature is separately meaningful (e.g., see Li et al., 2018). For structured data, decision trees, along with scoring systems, tend to be popular f”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, background): “, images from the training set that look similar to the image being classified [2, 6, 27, 32].”
Computing Optimal Decision Sets with SAT (2020, background): “Hence the rising interest in explainable artificial intelligence (XAI) [4, 14, 16, 18, 22, 23, 26, 29, 30, 36–38, 42, 43, 48, 49, 52], including research programs [3, 24] and legislation [17, 21].”
Concept whitening for interpretable image recognition (2020, methodology): “For example, neural networks have been designed to perform case-based reasoning (Chen et al., 2018; Li et al., 2018), to incorporate logical or grammatical structures (Li et al., 2017; Granmo et al., 2019; Wu and Song, 2019), to do clas-siﬁcation based on hard attention (Mnih et al., 2014; Ba et…”
Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition (2019, methodology): “…deep-learning models can be constructed for computer vision and time-Harvard series analysis (e.g., Chen et al., 2019; Y. Li et al., 2017; O. Li et al., 2018; Ming et al., 2019), then the standard should be changed from the assumption that interpretable models do not exist, to the…”
Interpretability Beyond Classification Output: Semantic Bottleneck Networks (2019, neutral): “Similar to our proposed approach, [1, 20] and [16] embed an interpretable layer into the network, yet are different in the way that they treat the concept representations as free parameters and train them jointly with the network parameters.”
Interpretable and Steerable Sequence Learning via Prototypes (2019, methodology): “Recently the concept has been incorporated in convolutional neural networks to build interpretable image classifiers [6, 19].”
This looks like that: deep learning for interpretable image recognition (2018, background): “[28] recently took the concepts in [23] and the preprint of an earlier version of this work, which both involve integrating prototype learning into CNNs for image recognition, and used these concepts to develop prototype learning in recurrent neural networks for modeling sequential data.”
Learning Optimized Risk Scores (2016, background): “…build models that obey constraints related to, for example, interpretability (see e.g. Caruana et al., 2015; Gupta et al., 2016; Chen et al., 2018; Li et al., 2018, where interpretability is addressed through constraints on model form), safety (Amodei et al., 2016), credibility (Wang et al.,…”
Towards Robust Interpretability with Self-Explaining Neural Networks (2018, background): “, for structured data), identifiability [12] and computational cost (with some of these methods requiring a full-fledged optimization subroutine [24]).”
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet (2019, methodology): “In Li et al. (2017) an autoencoder architecture is combined with a shallow classiﬁer based on prototype representations.”