Evaluating Feature Importance Estimates

By Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim. 2018

104 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

WT5?! Training Text-to-Text Models to Explain their Predictions (2020, background): “However, the reliability of some of these methods has been questioned (Kindermans et al., 2019; Hooker et al., 2018; Jain & Wallace, 2019; Serrano & Smith, 2019; Pruthi et al., 2019), limiting their practical util-”
Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning (2019, background): “…work has evaluated the applicability of saliency maps for explaining the behavior of image classiﬁers (Samek et al., 2017; Adebayo et al., 2018; Kindermans et al., 2019), but there is not a corresponding literature evaluating the applicability of saliency maps for explaining RL agent behavior.”
Aggregating explainability methods for neural networks stabilizes explanations (2019, methodology): “…is reﬂected in a lot of recent work into the problem (Kindermans et al., 2017; Selvaraju et al., 2017; Bach et al., 2015; Zhang et al., 2018; Zhou et al., 2016; Ancona et al., 2018; Ribeiro et al., 2016; Rieger et al., 2018; Kim et al., 2018; Lundberg & Lee, 2017; Zintgraf et al., 2017).”
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey (2019, background): “Some metrics to evaluate explainers are proposed, e.g. [80], but unfortunately, many of them tend to suffer from major drawbacks like computational cost [111] or simply focusing on one desirable attribute of a good explainer [112].”
Explainable AI for Trees: From Local Explanations to Global Understanding (2019, methodology): “We considered multiple previous approaches and based these metrics off what we considered the best aspects of prior evaluations [1, 25, 41, 65].”
Full-Gradient Representation for Neural Network Visualization (2019, methodology): “Second, we use the remove and retrain procedure [8] to evaluate saliency maps on the CIFAR100 dataset.”
Bridging Adversarial Robustness and Gradient Interpretability (2019, methodology): “…and its variants have been thoroughly studied in the context of attribution methods (Bach et al., 2015; Samek et al., 2017; Adebayo et al., 2018; Hooker et al., 2018).3 Since the loss gradient highlights the input features which affect the loss most strongly, and thus are important for the…”
Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents (2019, background): “[18] show that saliency methods do not produce analytically correct explanations for linear models and further reliability issues are discussed in [2, 13, 17].”
Explaining Motion Relevance for Activity Recognition in Video Deep Learning Models (2020, methodology): “brought to light when the decisions made by these models are explained using widely-used 2D saliency map methods, from the image recognition domain, and extended to 3D domain [8, 9, 10, 11, 12, 13, 14].”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, background): “Recent work has also focused on evaluations of explanations, ranging from human-centric evaluations [2, 5, 26] to functionally-grounded evaluations [27, 28, 29, 30, 10, 31, 32, 33].”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “However, in a different view [38, 49] that “without re-training, it is unclear whether the degradation in model performance”
Free-Lunch Saliency via Attention in Atari Agents (2019, methodology): “The current state of the art approaches are SmoothGrad [35] and VarGrad [1], which were studied theoretically in [32] and found to be best in a recent comprehensive evaluation [20].”