Explanatory Interactive Machine Learning

By Stefano Teso, Kristian Kersting. 2019

264 citations recorded by the source corpus.

Although interactive learning puts the user into the loop, the learner remains mostly a black box for the user. Understanding the reasons behind predictions and queries is important when assessing how the learner works and, in turn, trust. Consequently, we propose the novel framework of explanatory interactive learning where, in each step, the learner explains its query to the user, and the user interacts by both answering the query and correcting the explanation. We demonstrate that this can boost the predictive and explanatory powers of, and the trust into, the learned model, using text (e.g. SVMs) and image classification (e.g. neural networks) experiments as well as a user study.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, background): “A more interpretable direction is to associate features with specific labels as a source of supervision [32], or let users suggest adjustments of low-level features by examining model explanations [24, 43, 26].”
Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation (2020, background): “The framework explanatory interactive learning [43] also leverages human visual explanation with data-augmentation.”
Making deep neural networks right for the right scientific reasons by interacting with their explanations (2020, methodology): “A preliminary version of this manuscript has been published as a conference paper [16].”
Measuring the Quality of Explanations: The System Causability Scale (SCS) (2019, background): “Among the most important reasons is trust in the results which is improved by an explanatory interactive learning framework, where the algorithm is able to explain each step to the user and the user can interactively correct the explanation [15].”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “– enable the user to perform corrections on labels and to act CAIPI [129]”
Large pre-trained language models contain human-like biases of what is right and wrong to do (2021, background): “Here, explanatory interactive learning [47, 48, 49] is promising as it includes a mechanism enabling the AI system to explain its’ choices as well as a revision based on these explanations.”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, methodology): “In the case of creating feedback for a visual explanation, as in [43], [50] and [44], the mapping is quite clear: the user gives visual feedback denoting which regions of the input are relevant and which are not.”
Human-in-the-loop Debugging Deep Text Classifiers (2020, background): “Teso and Kersting (2019) proposed CAIPI, which is an explanatory interactive learning framework.”
Explanation-Based Human Debugging of NLP Models: A Survey (2021, background): “, 2016), and creating augmented training examples to reduce the effects of irrelevant words (Teso and Kersting, 2019).”