Learning Important Features Through Propagating Activation Differences

By Avanti Shrikumar, Peyton Greenside, Anshul B Kundaje. 2017

4,716 citations recorded by the source corpus.

The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, code: http://goo.gl/RM8jvH.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Yoshua Bengio
  3. Percy Liang
  4. Jianfeng Gao
  5. Noah A. Smith
  6. Luke S. Zettlemoyer
  7. Graham Neubig
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Andrew Zisserman
  7. Jitendra Malik
  8. Andrea Vedaldi

In Safety, Robustness & Interpretability

  1. Wojciech Samek
  2. K. Müller
  3. Been Kim
  4. Longo Luca
  5. Giulia Vilone
  6. Grgoire Montavon
  7. Su-In Lee
  8. Lars Kai Hansen

Citations in the corpus