Towards better understanding of gradient-based attribution methods for Deep Neural Networks

By Marco Ancona, Enea Ceolini, Cengiz Oztireli, Markus H. Gross. 2017

1,034 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Explaining Models by Propagating Shapley Values of Local Components (2019, background): “First, a gradient based interpretation of the Rescale rule has been discussed in (Ancona et al. 2018).”
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “…& Hansen, 2019), decomposition-based (Murdoch & Szlam, 2017; Shrikumar et al., 2016; Bach et al., 2015) and others (Dabkowski & Gal, 2017; Fong & Vedaldi, 2017; Ribeiro et al., 2016; Zintgraf et al., 2017), with many similarities among the meth-ods (Ancona et al., 2018; Lundberg & Lee, 2017).”
Sanity Checks for Saliency Maps (2018, methodology): “Ancona [24] showed that for ReLU networks (with zero baseline and no biases) the -LRP and DeepLift (Rescale) explanation methods are equivalent to the input gradient.”
On the Art and Science of Machine Learning Explanations (2018, methodology): “Popular techniques have names like DeepLIFT and integrated gradients [26], [27], [2].”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, methodology): “ng as expected in terms of overall logic ow, and 2) explain which aspects of the input data had an eect that lead to the specic output. 7 5.2.2 Attribution methods Attribution, a term introduced by Ancona et al. (2018), also referred to as relevance (Bach et al., 2015; Binder et al., 2016; Zintgraf ”
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet (2019, methodology): “The masking locations are determined by the heatmaps of BagNets which compare against random maskings as well as several popular attribution techniques (Baehrens et al., 2010; Sundararajan et al., 2017; Kindermans et al., 2018; Shrikumar et al., 2017) (we use the implementations of DeepExplain (Anco”
Explanations can be manipulated and geometry is to blame (2019, methodology): “These methods cover two classes of attribution methods, namely gradient-based and propagationbased explanations, and are frequently used in practice [26, 27].”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, methodology): “Common approaches to explainability of deep learning methods in computer vision are saliency maps [1], attention maps [47], and activation vectors [25].”
A Benchmark for Interpretability Methods in Deep Neural Networks (2018, background): “Our work considers several large scale datasets, whereas all previous evaluations have involved a far smaller subset of data [3, 29].”
Evaluating Feature Importance Estimates (2018, methodology): “(2017) with subsequent variations (Ancona et al., 2017; Fong & Vedaldi, 2017; Kindermans et al., 2017).”
Definitions, methods, and applications in interpretable machine learning (2019, background): “One line of work focuses on providing an overview of different interpretation methods with a strong emphasis on post hoc interpretations of deep learning models (8, 9), sometimes pointing out similarities between various methods (10, 11).”
On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. (2020, methodology): “In this sense, assessing their reliability via simple yet effective tests (22), understanding their common patterns and unique strengths (19), and seeking to unify them into a theoretically sound framework (48) are important research avenues to ensure that interpretability methods can be trusted…”
Techniques for interpretable machine learning (2018, methodology): “ck-propagating the relevance of the nal prediction score to the input layer [3]. These methods are integrated into a unied framework where all methods are reformulated as a modied gradient function [2]. This unication enables comprehensive comparison between dierent methods and facilitates eective i”
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications (2021, methodology): “Both the DeepExplain [4] and iNNvestigate [3] toolboxes built on top of the popular Keras [30] package for Python with TensorFlow backend for explaining Deep Neural Network models, and thus provide support for both CPU and GPU hardware and convenient access for users of Keras models.”
Transformation Importance with Applications to Cosmology (2020, background): “Thus far, a large majority of interpretability work has focused on attributing importance to raw features, such as pixels in an image or words in a document [5–14], with many similarities among the methods [15, 16].”
iNNvestigate neural networks! (2018, background): “In contrast, current related work (Kotikalapudi et al., 2017; Ancona et al., 2018) is limited to gradient-based methods.”
You Shouldn't Trust Me: Learning Models Which Conceal Unfairness From Multiple Explanation Methods (2020, methodology): “We use the authors’ repositories of SHAP and LIME and [4]’s implementation for the remaining methods.”