SmoothGrad: removing noise by adding noise

By Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda B. Viégas, Martin Wattenberg. 2017

2,674 citations recorded by the source corpus.

Explaining the output of a deep network remains a challenge. In the case of an image classifier, one type of explanation is to identify pixels that strongly influence the final decision. A starting point for this strategy is the gradient of the class score function with respect to the input image. This gradient can be interpreted as a sensitivity map, and there are several techniques that elaborate on this basic idea. This paper makes two contributions: it introduces SmoothGrad, a simple method that can help visually sharpen gradient-based sensitivity maps, and it discusses lessons in the visualization of these maps. We publish the code for our experiments and a website with our results.

Subfields: Artificial Intelligence, Biophysics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Safety, Robustness & Interpretability

Citations in the corpus

High Fidelity Visualization of What Your Self-Supervised Representation Knows About (2021, neutral): “Deterministic visualization methods: Many early works (Erhan et al., 2009; Zeiler & Fergus, 2014; Simonyan et al., 2013; Selvaraju et al., 2016; Smilkov et al., 2017) used gradient based techniques to visualize what is learned by neural networks.”
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, background): “To this end, 14 human judges 8 annotated 50 samples from the SST-2 validation set of sentiment excerpts (Socher et al., 2013).”
WT5?! Training Text-to-Text Models to Explain their Predictions (2020, background): “C L ] 3 0 A pr 2 02 0 Smilkov et al., 2017) or by designing architectures that are easier to analyze (Foerster et al., 2017; Jacobsen et al., 2018; Bahdanau et al., 2014; Raffel et al., 2017).”
No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World (2017, methodology): “Using a method similar to SmoothGrad [5], we looked at saliency maps to determine which parts of the images were most depended on by the model when making these classiﬁcations.”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Examples are LRP [40], DeepLIFT [41], CAM [42], Grad-CAM [43], Integrated gradients [44], and SmoothGrad [45].”
Analyzing Federated Learning through an Adversarial Lens (2018, background): “, 2017)), or generating sensitivity and saliency maps (SmoothGrad ((Smilkov et al., 2017)), Gradient Saliency Maps ((Simonyan et al.”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “Smilkov et al. (2017) observed that averaging a classifier’s input gradients over Gaussian corruptions of an image yields very interpretable saliency maps.”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “c to measure the quality of signal estimators. And then it proposes PatternNet and PatternAttribution that backpropagate parameters optimized against ˆ, resulting in saliency maps as well. SmoothGrad [60] improves gradient-based techniques by adding noises. Do visit the related website that displays”
Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks (2019, background): “SmoothGrad [40] and VarGrad [1] visually sharpen explanations by combining multiple explanations of noisy copies of the image.”
Sanity Checks for Saliency Maps (2018, background): “SmoothGrad (SG) [16] seeks to alleviate noise and visual diffusion [14, 13] for saliency maps by averaging over explanations of noisy copies of an input.”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “Also called sensitivity maps, saliency maps, or pixel attribution maps [92], these approaches use occlusion techniques or calculations with gradients to assign an ‘‘importance’’ value to individual pixels which are meant to reflect their influence on the final classification.”
Global-and-local attention networks for visual recognition (2018, methodology): “We visualized these features by calculating “smoothed” gradient images [44], which suppresses visual noise in gradient images.”
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge (2018, methodology): “An alternative is attribution methods that visualize how different parts of the input contribute to the ﬁnal output, such by generating sensitivity maps over the input (Bach et al., 2015; Baehrens et al., 2010; Selvaraju et al., 2016; Smilkov et al., 2017).”
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values (2018, background): “SmoothGrad (SG) (Smilkov et al., 2017) seeks to alleviate noise and visual diffusion (Sundararajan et al., 2017; Shrikumar et al., 2016) for saliency maps by averaging over explanations of noisy copies of an input.”
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2017, methodology): “Saliency methods are one of the most popular local explanation methods for image classification (Erhan et al., 2009; Smilkov et al., 2017; Selvaraju et al., 2016; Sundararajan et al., 2017; Dabkowski & Gal, 2017).”
Learning how to explain neural networks: PatternNet and PatternAttribution (2017, methodology): “The attribution is visualized as a heat map of pixel-wise contributions to the output principle include saliency maps from network gradients (Baehrens et al., 2010; Simonyan et al., 2014), DeConvNet (Zeiler & Fergus, 2014, DCN), Guided BackProp (Springenberg et al., 2015, GBP), Layer-wise Relevance ”
A unified view of gradient-based attribution methods for Deep Neural Networks (2017, background): “Nevertheless, Saliency Maps are usually rather noisy [15, 12, 21] and taking the absolute value prevents the detection of positive and negative evidence that might be present in the input.”
Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning (2021, methodology): “These methods include Integraded Gradients (Sundararajan et al., 2017), SmoothGrad (Smilkov et al., 2017), Guided GradCAM (Selvaraju et al., 2016), etc. Motivated by gradient explanation, DeepLIFT (Shrikumar et al., 2017) propose to use slope to describes how y changes as x differs from the baseline”