Explaining nonlinear classification decisions with deep Taylor decomposition

By Grgoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, Klaus‐Robert Müller . 2015

892 citations recorded by the source corpus.

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets. HighlightsA novel method to explain nonlinear classification decisions in terms of input variables is introduced.The method is based on Taylor expansions and decomposes the output of a deep neural network in terms of input variables.The resulting deep Taylor decomposition can be applied directly to existing neural networks without retraining.The method is tested on two large-scale neural networks for image classification: BVLC CaffeNet and GoogleNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “One of the representative works in this area is [60] , which presents a method to decompose the network classiﬁcation decision into contributions of its input elements.”
SNAS: Stochastic Neural Architecture Search (2018, methodology): “A full derivation is provided in Appendix D. Apparently, the search gradient is equivalent to a policy gradient for distribution at this edge whose credit is assigned as From a decision-wise perspective, this reward could be interpreted as contribution analysis of L with Taylor Decomposition (Montav”
Analyzing Federated Learning through an Adversarial Lens (2018, methodology): “These include (i) identifying the relevant features in the input pixel space for a particular decision via Layerwise Relevance Propagation (LRP) techniques ((Montavon et al., 2015)); (ii) visualizing the association between neuron activations and image features (Guided Backprop ((Springenberg et al.”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, background): “A recent series of work 13,17 – 22 has now begun to explain the predictions of non-linear ML methods in a wide set of complex real-world problems (e.g. 26 ).”
Visualizing and Understanding Atari Agents (2017, methodology): “These variants include Guided Backpropagation [24], Excitation Backpropagation [28], DeepLIFT [21], and Deep Taylor Series [17].”
Investigating the influence of noise and distractors on the interpretation of neural networks (2016, neutral): “The z -rule was originally proposed in [1] and later integrated in the deep Taylor framework [4].”
A Disentangling Invertible Interpretation Network for Explaining Latent Representations (2020, background): “Layer-wise relevance propagation [37] is a more general approach to propagate relevance scores through a network based on rules to distribute the relevance among input neurons.”
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions (2018, background): “1Layer-wise Relevance Propagation/Deep Taylor Expansion [45], decomposes the innards of deep learning systems to provide similar results to LIME (without directionality), and is also relevant here.”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, methodology): “Deep-Taylor decomposition [62] and integrated gradients [63] are famous attribution methods.”
Sequential Explanations with Mental Model-Based Policies (2020, methodology): “We compared three experimental policies in selecting explanations generated by prototype (Kim et al., 2016; Gurumoorthy et al., 2017) and deep Taylor decomposition saliency map (Montavon et al., 2017) explainers, relative to random selection baselines.”
iNNvestigate neural networks! (2018, background): “The corresponding Python code is: 1 import innvestigate 2 model = create a keras model () 3 analyzer = innvestigate . create analyzer (”analyzer name” , model) 4 analyzer . f i t ( X train ) # i f needed 5 analysis = analyzer . analyze ( X test ) Implemented methods At publication time the following”
Evaluating Feature Importance Estimates (2018, methodology): “Attribution estimators assign importance to input features by decomposing the output activation Aln into contributions from the individual input features (Bach et al., 2015; Sundararajan et al., 2017; Montavon et al., 2017; Shrikumar et al., 2016; Kindermans et al., 2017).”
Aggregating explainability methods for neural networks stabilizes explanations (2019, methodology): “In this paper, we focus on gradient-based explanation methods (Simonyan et al., 2013; Zeiler & Fergus, 2014; Selvaraju et al., 2017; Smilkov et al., 2017; Sundararajan et al., 2017; Shrikumar et al., 2017; Montavon et al., 2017).”
Interpretable and Differentially Private Predictions (2019, methodology): “One prominent aspect of these approaches is the use of gradient-based attributions (Selvaraju et al. 2016; Ribeiro, Singh, and Guestrin 2016; Smilkov et al. 2017; Sundararajan, Taly, and Yan 2017; Montavon et al. 2015; Bach et al. 2015; Ancona et al. 2017).”
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration (2022, background): “Specifically, given the attention maps A k for each transformer block k , and relevance scores R k [45], the saliency map L ViT c is given by: where E h computes the mean over the attention heads and ⊙ represents the element-wise product.”
Interrogating the Explanatory Power of Attention in Neural Machine Translation (2019, background): “Visualizing and interpreting neural models has been extensively studied in computer vision (Simonyan et al., 2013; Bach et al., 2015; Zeiler and Fergus, 2014; Montavon et al., 2017), and more recently in natural language processing (NLP) (Karpathy et al.”