Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing

By Sarah Wiegreffe, Ana Marasovic. 2021

168 citations recorded by the source corpus.

Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, background): “Because it is necessary that annotators understand the task before they can evaluate explanation quality (Wiegreffe and Marasovi´c, 2021), for tasks that are more difﬁcult, i.e., NLI, we additionally require annotators to pass (score of 7 or above) a task-speciﬁc qualiﬁcation exam with 8 questions,…”
Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “However, using dataset-collection to explicitly collect sufﬁcient rationales does not address the unnaturalness of such a task formulation (Wiegreffe and Marasovi´c, 2021).”
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022, methodology): “In understanding advances in technologies, a diferent approach has emerged: developing interaction datasets and providing tools for data analyses [17, 51, 54, 61, 67].”
Case-Based Abductive Natural Language Inference (2020, background): “In the context of Natural Language Inference (NLI), this task is often used to develop and evaluate explainable systems, capable of performing transparent multi-step reasoning with natural language (Wiegreffe and Marasović, 2021; Thayaparan et al., 2020).”
NAREOR: The Narrative Reordering Problem (2021, background): “…understand them since the order implies not only temporal but other inferences about causality, processes of change, etc. Narrative order can thus inﬂuence model explainability , especially for explanation generation (Rajani et al. 2019), a recent area-of-interest (Wiegreffe and Marasovic 2021).”
REV: Information-Theoretic Evaluation of Free-Text Rationales (2022, background): “In practice, vacuous rationales such as ˆ r 1 ,a are commonly seen in model generations (Sun et al., 2022; Wiegreffe and Marasovic, 2021).”
Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations (2022, background): “Prior work on explainable NLP (Wiegreffe and Marasovic, 2021) has explored different forms of explanations ranging from extractive rationales (Zaidan et al.”
On the Diversity and Limits of Human Explanations (2021, background): “, E2 in Table 1) and directly maps to highlights in Wiegreffe and Marasović (2021). However, it does not map to any existing definitions of explanations in the psychology literature since”
Post-hoc Interpretability for Neural NLP: A Survey (2021, background): “This issue is even more problematic because the annotated explanations are provided by humans who have no insights into the model’s behavior [128].”
Decomposition Enhances Reasoning via Self-Evaluation Guided Decoding (2023, unknown): “One approach is to generate rationales to enhance model interpretability (Zhou et al., 2020; Wiegreffe and Marasović, 2021; Wiegreffe et al., 2021).”
Trustworthy AI: From Principles to Practices (2021, methodology): “To train the explanation task, it is helpful to consider collecting explanations or information that may not directly be part of the end task, either directly from annotators [354] or with the help of automated approaches [185].”