Explainable Deep Learning: A Field Guide for the Uninitiated

By Ning Xie, Gabrielle Ras, Marcel A. J. van Gerven, Derek Doran. 2020

452 citations recorded by the source corpus.

Deep neural networks (DNNs) are an indispensable machine learning tool despite the difficulty of diagnosing what aspects of a model’s input drive its decisions. In countless real-world domains, from legislation and law enforcement to healthcare, such diagnosis is essential to ensure that DNN decisions are driven by aspects appropriate in the context of its use. The development of methods and studies enabling the explanation of a DNN’s decisions has thus blossomed into an active and broad area of research. The field’s complexity is exacerbated by competing definitions of what it means “to explain” the actions of a DNN and to evaluate an approach’s “ability to explain”. This article offers a field guide to explore the space of explainable deep learning for those in the AI/ML field who are uninitiated. The field guide: i) Introduces three simple dimensions defining the space of foundational methods that contribute to explainable deep learning, ii) discusses the evaluations for model explanations, iii) places explainability in the context of other related deep learning research areas, and iv) discusses user-oriented explanation design and future directions. We hope the guide is seen as a starting point for those embarking on this research field.

Subfields: Artificial Intelligence, Health Informatics, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Natural Language Processing

Citations in the corpus

Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “, 2022) serves as a type of explanation (Ras et al., 2022), providing an insight into the external information the model was provided when generating the answer.”
Teaching language models to support answers with verified quotes (2022, background): “One could view self-supporting answers as a specific type of explanation, putting our work alongside other work in explainable AI (Ras et al., 2020) that aims to provide natural-language explanations of QA model responses (Lamm et al.”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, unknown): “Current methods can be divided into various categories based on characteristics [55], e.”
Tell me why! Explanations support learning relational and causal structure (2021, background): “That is, rather than seeking explanations post-hoc, to help humans understand a system (e.g., Chen et al., 2018; Topin & Veloso, 2019; Xie et al., 2020), these works use explanations to help a system understand a task (cf.”
SurvNAM: The machine learning survival model explanation (2021, background): “The importance of the explanation problem leads to developing many methods and models which try to explain and to interpret predictions of the machine learning models [2, 3, 4, 5, 6, 7, 8, 9, 10].”
Tell me why! - Explanations support learning of relational and causal structure (2021, background): “That is, rather than seeking explanations only posthoc, in an attempt to help humans understand the system (e.g. Chen et al., 2018; Topin and Veloso, 2019; Xie et al., 2020), these works use explanations to help the system understand the task (cf.”
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2022, methodology): “Since we could not find a complete and recent overview of explanation types, we reviewed recent XAI surveys [16, 59, 171, 287] and adapted and extended the categories identified by Guidotti et al.”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “, [141]; finite state automata as of [139]; simple clustering and nearest neighbors approaches as of [20];”