Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning

By Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael A. Specter, Lalana Kagal. 2018

187 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

On the Art and Science of Machine Learning Explanations (2018, background): “Others have deﬁned key terms and put forward general motivations for better interpretability of machine learning models (and advocated for stronger scientiﬁc rigor in certain cases) [9], [11], [13], [18].”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[154] proposed a methodological approach for evaluating interpretability of ML models according to a taxonomy that distinguishes three types of explainability: emulate the processing, explain the representation and explanation-producing networks.”
Understanding Multi-Head Attention in Abstractive Summarization (2019, background): “, 2015) has gained popularity as a means of obtaining insight in the inner workings of deep neural networks (e.g., Lei, 2017; Choi et al., 2016; Gilpin et al., 2018; Ghaeini et al., 2018).”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “[24] stated the term as a science of perceiving what a model did or might have done.”
The Price of Interpretability (2019, background): “However, even if these approximations are almost as accurate as the original model, they may have very different behavior on some inputs and can thus provide a misleading assessment of the model’s behavior [18].”
Sequential Explanations with Mental Model-Based Policies (2020, background): “Other studies used (or supported the use of) similar metrics as proxies for inter-pretability (Doshi-Velez & Kim, 2017; Lipton, 2016; Gilpin et al., 2018; Madumal et al., 2019; Poursabzi-Sangdeh et al., 2018; Lahav et al., 2018).”
Definitions, methods, and applications in interpretable machine learning (2019, background): “Other work has focused on the narrower problem of how interpretations should be evaluated (11, 12) and what properties they should satisfy (13).”
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, methodology): “Moreover, the usefulness of an explanation depends on the reason to demand explainability (more on this in Sections 3 and 6) [24, 26].”
Spatial Temporal Transformer Network for Skeleton-based Action Recognition (2020, background): “A great AI debate of the last years is interpretability [57], the ability of one model to provide reasonable explanations”
Explaining Explanations to Society (2019, methodology): “We follow from previous work [10] that a proper explanation should be both interpretable and complete.”
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (2019, background): “Recent work argues that the sophistication and completeness of both interpretability and explanations depends on the audience [20, 54].”
How Case-Based Reasoning Explains Neural Networks: A Theoretical Analysis of XAI Using Post-Hoc Explanation-by-Example from a Survey of ANN-CBR Twin-Systems (2019, methodology): “Attempts to make ANNs more interpretable use many different “explanation methods” that are often specific to a given architecture (see reviews [5, 9, 37]).”
The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic Decision Making Systems (2021, unknown): “For an overview and discussion see (Gilpin et al. 2018; Mehrabi et al. 2019).”