TED: Teaching AI to Explain its Decisions

By Noel C. F. Codella, Michael Hind, Karthikeyan Natesan Ramamurthy, Murray Campbell, Amit Dhurandhar, Kush R. Varshney, Dennis Wei, Aleksandra Mojsilovic. 2018

115 citations recorded by the source corpus.

Artificial intelligence systems are being increasingly deployed due to their potential to increase the efficiency, scale, consistency, fairness, and accuracy of decisions. However, as many of these systems are opaque in their operation, there is a growing demand for such systems to provide explanations for their decisions. Conventional approaches to this problem attempt to expose or discover the inner workings of a machine learning model with the hope that the resulting explanations will be meaningful to the consumer. In contrast, this paper suggests a new approach to this problem. It introduces a simple, practical framework, called Teaching Explanations for Decisions (TED), that provides meaningful explanations that match the mental model of the consumer. We illustrate the generality and effectiveness of this approach with two different examples, resulting in highly accurate explanations with no loss of prediction accuracy for these two examples.

Subfields: Artificial Intelligence, Information Systems and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems and Management

Citations in the corpus

Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees (2020, background): “In other areas such as flight control or self driving cars, DNNs are sidelined (at least for mass deployment) by their opaqueness or lack of decision making interpretability (Bastani, Kim, and Bastani 2017), (Hind et al. 2019).”
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques (2019, methodology): “The TED framework [30] requires a training dataset that includes explanations ( E ) in addition to features ( X ) and labels ( Y ).”
Evaluating and Aggregating Feature-based Model Explanations (2020, methodology): “Note we omit evaluation criteria that assume access to ground-truth explanations for training points; for a thorough treatment on this topic, see [Hind et al. , 2019; Osman et al. , 2020].”
Benchmarking and survey of explanation methods for black box models (2021, methodology): “Another interesting method proposed in this toolkit is ted (Hind et al. 2019; Dash et al. 2018), which provides intrinsic local explanations and provides global explanations based on rules.”
What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research (2021, background): “Among others, users take into account recommendations of artificial systems to make decisions [24].”
Expanding Explainability: Towards Social Transparency in AI systems (2021, background): “Moreover, a notable area of XAI work focuses on generating human-consumable and domain-specific machine explanations by learning from how humans explain [27, 40], which could be a fruitful area to explore when combined with the availability of 4W information.”
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, methodology): “e) model for the subset of data where the interpretable model has unsatisfactory performance and Papernot and McDaniel [58] use K-nearest neighbors in each layer of a deep neural network. Hind et al. [59] use a training set annotated with explanations to train the task model to provide explanations.”
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences (2021, background): “There is an emerging area of XAI research on generating human-consumable explanations with the supervision of human explanations [34, 50, 56], which essentially translates model reasoning into meaningful human explanations applied to the same prediction.”