Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI

By Shane T. Mueller, Robert R. Hoffman, William J. Clancey, Abigail Emrey, Gary Klein. 2019

323 citations recorded by the source corpus.

This is an integrative review that address the question, "What makes for a good explanation?" with reference to AI systems. Pertinent literatures are vast. Thus, this review is necessarily selective. That said, most of the key concepts and issues are expressed in this Report. The Report encapsulates the history of computer science efforts to create systems that explain and instruct (intelligent tutoring systems and expert systems). The Report expresses the explainability issues and challenges in modern AI, and presents capsule views of the leading psychological theories of explanation. Certain articles stand out by virtue of their particular relevance to XAI, and their methods, results, and key points are highlighted. It is recommended that AI/XAI researchers be encouraged to include in their research reports fuller details on their empirical or experimental methods, in the fashion of experimental psychology research reports: details on Participants, Instructions, Procedures, Tasks, Dependent Variables (operational definitions of the measures and metrics), Independent Variables (conditions), and Control Conditions.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, background): “Moreover, the usefulness of an explanation depends on the reason to demand explainability (more on this in Sections 3 and 6) [24, 26].”
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, methodology): “General Towards more formalism [1, 31, 10, 32, 9, 33, 26, 2, 34, 8, 35, 36, 37, 18, 38, 39, 40], [41, 42, 43, 44, 45, 46] ∗ Multidisciplinary research collaborations [47, 35, 18, 48, 49, 50], [45, 51, 42] * Explanations and the nature of user experience and expertise [28, 24, 52, 26, 39, 2], [43, 41”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “The same holds for the very detailed DARPA report [95] from 2019.”
A historical perspective of explainable Artificial Intelligence (2020, background): “…in machine learning and Deep Learning (Arrieta et al., 2020; Fernandez, Herrera, Cordon, Jose del Jesus, & Marcelloni, 2019; Guidotti et al., 2018; Mueller, Hoffman, Clancey, Emrey, & Klein, 2019), recommender systems (Nunes & Jannach, 2017; Tintarev & Masthof, 2015), and Neural-Symbolic…”
What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research (2021, background): “beyond the scope of the current paper (see, for instance, [144] for a review on understanding, [146, 147] for papers on the concept of understanding, [138, 148, 149] for the relation between explanations and understanding, or [150] for the related topic of cognitive processes in knowledge acquisitio”
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI (2021, methodology): “Some researchers speculated that an IOED may be at play when users deal with explanations from XAI systems [14, 35, 54].”
Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI (2021, background): “…it is necessary to overcome the unilateral evaluation methods in some existing studies that only evaluate the performance of AI systems, HCI should promote the evaluation of the AI systems as a human-machine system by adding an end users perspective (Hoffman et al., 2018; Mueller et al., 2019).”
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams (2019, background): “Recently, the research on explainable AI (XAI) has expanded rapidly (e.g. see [23,51,104]).”
Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale (2022, background): “Owing to the complexity and black-box nature of AI (Mueller et al. 2019), the results produced by it require careful consideration and evaluation.”