Human-grounded Evaluations of Explanation Methods for Text Classification

By Piyawat Lertvittayakumjorn, Francesca Toni. 2019

73 citations recorded by the source corpus.

Due to the black-box nature of deep learning models, methods for explaining the models’ results are crucial to gain trust from humans and support collaboration between AIs and humans. In this paper, we consider several model-agnostic and model-specific explanation methods for CNNs for text classification and conduct three human-grounded evaluations, focusing on different purposes of explanations: (1) revealing model behavior, (2) justifying model predictions, and (3) helping humans investigate uncertain predictions. The results highlight dissimilar qualities of the various explanation methods we consider and show the degree to which these methods could serve for each purpose.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, background): “sented with an explanation and an input, and must correctly predict the model’s output (Doshi-Velez and Kim, 2017; Lertvittayakumjorn and Toni, 2019; Hase and Bansal, 2020).”
A Diagnostic Study of Explainability Techniques for Text Classification (2020, background): “We further find that the CNN model’s confidence is better predicted with InputXGradµ.”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “Lertvittayakumjorn and Toni [118] also present three user tasks to evaluate local explanation techniques for text classification through revealing model behavior to human users, justifying the predictions, and helping humans investigate uncertain predictions.”
Learning to Faithfully Rationalize by Construction (2020, background): “Lertvittayakumjorn and Toni (2019) classify three possible uses of text explanations: ( i ) revealing model behavior, ( ii ) justifying model predictions, and ( iii ) helping humans investigate uncertain predictions.”
A Survey of the State of Explainable AI for Natural Language Processing (2020, background): “While explainability may generally serve many purposes (see, e.g., Lertvittayakumjorn and Toni, 2019), our focus is on explainability from the perspective of an end user whose goal is to understand how a model arrives at its result, also referred to as the outcome explanation problem (Guidotti et…”
Human-in-the-loop Debugging Deep Text Classifiers (2020, methodology): “This is similar to how previous works analyze CNN features (Jacovi et al., 2018; Lertvittayakumjorn and Toni, 2019), and it is equivalent to back-propagating the feature values to the input using LRP and cropping the consecutive input words with non-zero LRP scores to show in the word clouds.”
Explanation-Based Human Debugging of NLP Models: A Survey (2021, background): “, 2016), this is not easy when the model flaws are not obvious (Zhang et al., 2020; Lertvittayakumjorn and Toni, 2019).”