Understanding Black-box Predictions via Influence Functions

By Pang Wei Koh, Percy Liang. 2017

3,706 citations recorded by the source corpus.

How can we explain the predictions of a black-box model? In this paper, we use influence functions — a classic technique from robust statistics — to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Percy Liang
  3. Yoshua Bengio
  4. Luke S. Zettlemoyer
  5. Jianfeng Gao
  6. Yejin Choi
  7. Quoc V. Le
  8. Noah A. Smith

In Natural Language Processing

  1. Mohit Bansal
  2. Yejin Choi
  3. Luke S. Zettlemoyer
  4. Percy Liang
  5. Noah A. Smith
  6. Hannaneh Hajishirzi
  7. Graham Neubig
  8. Jianfeng Gao

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Nicolas Papernot
  3. Eric Wallace
  4. Bo Li
  5. Florian Tramèr
  6. Percy Liang
  7. Matthias Hein
  8. Longo Luca

Citations in the corpus