On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

By Sebastian Bach, Alexander Binder, Grgoire Montavon, Frederick Klauschen, K. Müller, Wojciech Samek. 2015

4,977 citations recorded by the source corpus.

Understanding and interpreting classification decisions of automated image classification systems is of high value in many applications, as it allows to verify the reasoning of the system and provides additional information to the human expert. Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision. This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers. We introduce a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks. These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest. We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Mohit Bansal
  3. Yonatan Belinkov
  4. Percy Liang
  5. Quoc V. Le
  6. Jianfeng Gao
  7. Luke S. Zettlemoyer
  8. Kyunghyun Cho

In Biophysics

  1. Wojciech Samek
  2. K. Müller
  3. Grgoire Montavon
  4. Been Kim
  5. Sebastian Lapuschkin
  6. Lars Kai Hansen
  7. Longo Luca
  8. Lior Wolf

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Andrea Vedaldi
  6. Xiaogang Wang
  7. Jitendra Malik
  8. Andrew Zisserman

In Safety, Robustness & Interpretability

  1. Wojciech Samek
  2. K. Müller
  3. Longo Luca
  4. Giulia Vilone
  5. Been Kim
  6. Grgoire Montavon
  7. Su-In Lee
  8. Scott M. Lundberg

Citations in the corpus