An Empirical Comparison of Instance Attribution Methods for NLP

By Pouya Pezeshkpour, Sarthak Jain, Byron C. Wallace, Sameer Singh. 2021

45 citations recorded by the source corpus.

Widespread adoption of deep models has motivated a pressing need for approaches to interpret network outputs and to facilitate model debugging. Instance attribution methods constitute one means of accomplishing these goals by retrieving training instances that (may have) led to a particular prediction. Influence functions (IF; Koh and Liang 2017) provide machinery for doing this by quantifying the effect that perturbing individual train instances would have on a specific test prediction. However, even approximating the IF is computationally expensive, to the degree that may be prohibitive in many cases. Might simpler approaches (e.g., retrieving train examples most similar to a given test point) perform comparably? In this work, we evaluate the degree to which different potential instance attribution agree with respect to the importance of training samples. We find that simple retrieval methods yield training instances that differ from those identified via gradient-based methods (such as IFs), but that nonetheless exhibit desirable characteristics similar to more complex attribution methods. Code for all methods and experiments in this paper is available at: https://github.com/successar/instance_attributions_NLP.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Foundation Models and Fair Use (2023, neutral): “…attribution refers to methods that assign attribution scores to training examples to understand the contribution of individual examples (or group of examples) to (test-time) model predictions (Koh & Liang, 2017; Ghorbani & Zou, 2019; Jia et al., 2019; Pezeshkpour et al., 2021; Ilyas et al., 2022).”
A Review on Language Models as Knowledge Bases (2022, methodology): “Gradient-based and non-Hessian information inﬂuence attribution methods (Pezeshkpour et al., 2021b) have been introduced to speed up computation.”
Combining Feature and Instance Attribution to Detect Artifacts (2021, methodology): “Importance can reflect a formal approximation of the change in ŷt when zi is upweighted (Koh and Liang, 2017) or can be derived via heuristic methods (Pezeshkpour et al., 2021; Rajani et al., 2020).”
Datamodels: Predicting Predictions from Training Data (2022, methodology): “As in prior work (Hanawa et al., 2021; Pezeshkpour et al., 2021), we consider a representative set of baselines spanning both methods based on representation similarity and gradient-based methods, such as influence functions.”