e-SNLI: Natural Language Inference with Natural Language Explanations

By Oana-Maria Camburu, Tim Rocktaschel, Thomas Lukasiewicz, Phil Blunsom. 2018

766 citations recorded by the source corpus.

In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model’s decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, background): “, 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al.”
Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech (2023, background): “Moreover, it has been a challenging problem to provide the explainability of those models by using natural language explanations (NLEs) [4, 8].”
Scaling Instruction-Finetuned Language Models (2022, methodology): “, 2022, inter alia) or focuses on models of substantially smaller scale (Ling et al., 2017; Camburu et al., 2018; Rajani et al., 2019; Talmor et al., 2020; Zelikman et al., 2022).”
Improving Wikipedia verifiability with AI (2022, background): “…to the best search term for ﬁnding evidence—two long-standing problems that have been primarily investigated in somewhat synthetic settings (Bowman et al., 2015; Wang et al., 2018; Camburu et al., 2018; Nie et al., 2019; Pérez-Rosas et al., 2017; Thorne et al., 2018; Thorne & Vlachos, 2018).”
ExpBERT: Representation Engineering with Natural Language Explanations (2020, background): “Rather than using language to define a global collection of features, Rajani et al. (2019) and Camburu et al. (2018) use instance-level explanations to train models that generate their own explanations.”
Shaping Visual Representations with Language for Few-Shot Classification (2019, neutral): “…studies models which generate natural language explanations of decisions for interpretability for both textual (e.g. natural language inference; [3]) and visual [17, 18] tasks, but here we examine whether this act of predicting language can actually improve downstream task performance; similar…”
Large Language Models Can Self-Improve (2022, neutral): “For example, in the supervised learning setting, a model can be fine-tuned using human-annotated rationales (Zaidan et al., 2007; Ling et al., 2017a; Narang et al., 2020; Camburu et al., 2018; Cobbe et al., 2021; Chung et al., 2022).”
Training Language Models with Language Feedback (2022, methodology): “Prior work explores ways of using explanations to train text classiﬁcation models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Can language models learn from explanations in context? (2022, background): “…(e.g. Wei et al., 2022; Reynolds and McDonell, 2021), as well as a huge variety of broader work beyond prompting on topics like training or tuning with explanations (e.g. Camburu et al., 2018; Ling et al., 2017; Hendricks et al., 2016; Narang et al., 2020; Lertvittayakumjorn and Toni, 2021).”
STaR: Bootstrapping Reasoning With Reasoning (2022, background): “Natural Language Explanations Natural language explanations have also been discussed from the perspective of explainable machine learning, focusing on justification rather than reasoning [26, 27].”
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, methodology): “Crowdsourced free-text explanations are provided in the e-SNLI dataset (Camburu et al., 2018).”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, methodology): “We instantiate our few-shot learning approaches on e-SNLI (Camburu et al., 2018) as parent task, and WinoGrande (Sakaguchi et al., 2020) and ComVE (Wang et al., 2020) as child tasks.”
Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Note that we use T5 with the ≈ T5 prompt for E-SNLI, and U NI - FIED QA with QA SIMPLE (§3) for other datasets to establish the best possible performance for each dataset.”
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, neutral): “, 2019), e-SNLI (Camburu et al., 2018) and recently, Eraser benchmark (DeYoung et al.”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, background): “Meanwhile, natural language explanations have been treated as targets in a multitask, sequence-to-sequence framework, using datasets with free-form textual annotations for each data point (Camburu et al., 2018; Narang et al., 2020; Hase et al., 2020; Wiegreffe et al., 2020).”
Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “A smaller improvement for E-SNLI could be explained by the fact that E-SNLI has substantially more training examples for each label than CoS-E, which helps a pipeline model learn features predictive of each label.”
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables (2020, background): “[3] that interpretability comes at the cost of loss in performance.”
WT5?! Training Text-to-Text Models to Explain their Predictions (2020, methodology): “The primary differences between Camburu et al. (2018) and this work are that our approach requires no special model architecture and that we take advantage of a pre-trained model that is already quite capable of generating natural text.”