A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Values

By Mukund Sundararajan, Ankur Taly. 2018

22 citations recorded by the source corpus.

Local explanation methods, also known as attribution methods, attribute a deep network's prediction to its input (cf. Baehrens et al. (2010)). We respond to the claim from Adebayo et al. (2018) that local explanation methods lack sensitivity, i.e., DNNs with randomly-initialized weights produce explanations that are both visually and quantitatively similar to those produced by DNNs with learned weights. Further investigation reveals that their findings are due to two choices in their analysis: (a) ignoring the signs of the attributions; and (b) for integrated gradients (IG), including pixels in their analysis that have zero attributions by choice of the baseline (an auxiliary input relative to which the attributions are computed). When both factors are accounted for, IG attributions for a random network and the actual network are uncorrelated. Our investigation also sheds light on how these issues affect visualizations, although we note that more work is needed to understand how viewers interpret the difference between the random and the actual attributions.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Giuseppe Di Fatta
2. Mario Cannataro
3. Huizhi Liang
4. Chiara Zucco

Citations in the corpus

• Explainable Sentiment Analysis with Applications in Medicine (2018, background): “There are some conflicting views regarding the lackness in sensitivity of local explanation methods [55], [56].”