On the (In)fidelity and Sensitivity for Explanations.

By Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, Pradeep Ravikumar. 2019

604 citations recorded by the source corpus.

We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while the optimal explanation for sensitivity is a vacuous constant explanation, the optimal explanation for infidelity is a novel combination of two popular explanation methods. By varying the perturbation distribution that defines infidelity, we obtain novel explanations by optimizing infidelity, which we show to out-perform existing explanations in both quantitative and qualitative measurements. Another salient question given these measures is how to modify any given explanation to have better values with respect to these measures. We propose a simple modification based on lowering sensitivity, and moreover show that when done appropriately, we could simultaneously improve both sensitivity as well as fidelity.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Do Not Trust Additive Explanations (2019, background): “Such tools are widely adopted, but little is said about the quality of their explanations (Guidotti and Ruggieri, 2018; Yeh et al., 2019).”
Post-hoc Interpretability for Neural NLP: A Survey (2021, background): “However, in other cases, most notably input features, providing a functionally-grounded metric can be very challenging (Jacovi and Goldberg, 2020; Kindermans et al., 2017; Yeh et al., 2019; Adebayo et al., 2018; Hooker et al., 2019).”
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics (2021, background): “Sensitivity [84,85] is proposed to measure the degree to which the explanation is affected by insigniﬁcant perturbations from the test point.”
Evaluating and Aggregating Feature-based Model Explanations (2020, methodology): “High faithfulness appears as relevance in [ Samek et al. , 2016 ] , as gold set in [ Ribeiro et al. , 2016 ] , as faithfulness in [ Plumb et al. , 2018 ] , as sensitivity-n in [ Ancona et al. , 2018 ] , and as inﬁdelity in [ Yeh et al. , 2019 ] .”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “Researchers [29, 47, 81, 101, 107, 183] also proposed many other desiderata for interpretations, interpretation algorithms or interpretability, such as fairness, privacy, reliability, robustness, causality, trust, fidelity, faithfulness, transferability, informativeness, transparency, plausibility, ”
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, methodology): “The fragility of posthoc XAI methods to small perturbations at the model’s input and the known inconsistency in synthesized explanations for a given input [166] (see Section 3.2.3 above) highlight the challenge of creating compelling explanations.”
Debugging Tests for Model Explanations (2020, methodology): “, 2017); and finally, 3) We select methods that have been shown to have desirable against manipulation (SmoothGrad) (Dombrowski et al., 2019; Yeh et al., 2019).”
On Concept-Based Explanations in Deep Neural Networks (2019, background): “Among these forms of interpretability, different evaluations of explanations are proposed, including more human-centric evaluations [19, 15] and functionally-grounded evaluations [24, 14, 1, 31].”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, background): “Recent work has also focused on evaluations of explanations, ranging from human-centric evaluations [2, 5, 26] to functionally-grounded evaluations [27, 28, 29, 30, 10, 31, 32, 33].”
Captum: A unified and generic model interpretability library for PyTorch (2020, methodology): “To address these issues, we provide generic implementations of two evaluation metrics called inﬁdelity and max-sensitivity proposed in [19].”