Representer Point Selection for Explaining Deep Neural Networks

By Chih-Kuan Yeh, Joon Sik Kim, Ian En-Hsu Yen, Pradeep Ravikumar. 2018

313 citations recorded by the source corpus.

We propose to explain the predictions of a deep neural network, by pointing to the set of what we call representer points in the training set, for a given test point prediction. Specifically, we show that we can decompose the pre-activation prediction of a neural network into a linear combination of activations of training points, with the weights corresponding to what we call representer values, which thus capture the importance of that training point on the learned parameters of the network. But it provides a deeper understanding of the network than simply training point influence: with positive representer values corresponding to excitatory training points, and negative values corresponding to inhibitory points, which as we show provides considerably more insight. Our method is also much more scalable, allowing for real-time feedback in a manner not feasible with influence functions.

Subfields: Artificial Intelligence, Materials Chemistry

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

Citations in the corpus

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “The representer theorem is introduced for studying the extent of effect xi has on a decision made by a deep NN [129].”
Neural Prototype Trees for Interpretable Fine-grained Image Recognition (2020, methodology): “part-based methods [56, 61], saliency maps [6, 13, 60] or representer points [53].”
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation (2020, background): “Unlike some of the existing techniques [19, 30, 24] it is also completely model-agnostic and is itself easy to explain.”
GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks (2020, methodology): “Instead of creating surrogate models, some methods has been proposed to identify the patterns of input data and find influential samples for relevant information [7], [8].”
Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable (2016, background): “This experiment is similar to that ran by [25, 51].”
Revisiting Methods for Finding Influential Examples (2021, methodology): “…(Influence Functions, Grad-Dot, Grad-Cos, TraceIn and Representer Point Selection) for finding influential examples are unstable, albeit some more than others: Influence functions (Koh and Liang 2017) are extremely unstable, for example; Representer Point Selection (Yeh et al. 2018) less so.”
The Bouncer Problem: Challenges to Remote Explainability (2019, background): “Many works address explainability of in-spected model decisions on a local setup (please refer to surveys [8,13,20]) – some speciﬁcally for neural network models [31] –, where the number of requests to the model is unbounded.”
TRAK: Attributing Model Behavior at Scale (2023, unknown): “, 2022); as well as various other approaches (Yeh et al., 2018; Pruthi et al., 2020; Hammoudeh & Lowd, 2022b).”
Attention-Based Prototypical Learning Towards Interpretable, Confident and Robust Deep Neural Networks (2019, methodology): “(Yeh et al., 2018) decomposes the prediction into a linear combination of activations of training points for interpretability using representer values.”
Machine Unlearning for Random Forests (2020, background): “Instance-Based Interpretability A popular form of interpretability looks at how much each training instance contributes to a given prediction (Koh & Liang, 2017; Yeh et al., 2018; Sharchilev et al., 2018; Pruthi et al., 2020; Chen et al., 2021).”
Combining Feature and Instance Attribution to Detect Artifacts (2021, background): “…their “impor-tance” for an output (Ribeiro et al., 2016; Lund-berg and Lee, 2017; Adebayo et al., 2018), and; (2) instance attribution , which retrieves training instances most responsible for a given prediction (Koh and Liang, 2017; Yeh et al., 2018; Rajani et al., 2020; Pezeshkpour et al., 2021).”
Explainability for Large Language Models: A Survey (2023, background): “This family of approaches characterizes the influence of individual training sample by measuring how much they affect the loss on test points [Yeh et al. 2018].”
Interpreting Predictions of NLP Models (2020, methodology): “We will cover inﬂuence functions (Koh and Liang, 2017) and representor points (Yeh et al., 2018).”
DART: Data Addition and Removal Trees (2020, unknown): “…training instances caused by randomness or dataset poisoning attacks (Mozaffari-Kermani, Sur-Kolay et al. 2014; Steinhardt, Koh, and Liang 2017) which may be more efﬁciently identiﬁed using instance-attribution explanations (Koh and Liang 2017; Sharchilev et al. 2018; Yeh, Kim et al. 2018).”
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences (2021, unknown): “Without burdening people with the model decision complexities, showing representative examples and their predicted outcomes [83, 108], which overall have a good coverage representing the input space (whether constructed or extracted from the training data), can be used as example-based global explan”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, background): “For nonlinear models, influence function [75] and representer value [151] are proposed.”
Datamodels: Predicting Predictions from Training Data (2022, methodology): “, 2020) and metrics based on representation similarity (Charpiat et al., 2019; Yeh et al., 2018)—see (Hanawa et al.”
Attribute Prototype Network for Zero-Shot Learning (2020, methodology): “Some methods [3, 48, 24] base the network decision on several relevant examples i.”