When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data

By Peter Hase, Mohit Bansal. 2021

95 citations recorded by the source corpus.

Many methods now exist for conditioning models on task instructions and user-provided explanations for individual data points. These methods show great promise for improving task performance of language models beyond what can be achieved by learning from individual (x,y) pairs. In this paper, we (1) provide a formal framework for characterizing approaches to learning from explanation data, and (2) we propose a synthetic task for studying how models learn from explanation data. In the first direction, we give graphical models for the available modeling approaches, in which explanation data can be used as model inputs, as targets, or as a prior. In the second direction, we introduce a carefully designed synthetic task with several properties making it useful for studying a model’s ability to learn from explanation data. Each data point in this binary classification task is accompanied by a string that is essentially an answer to the why question: “why does data point x have label y?” We aim to encourage research into this area by identifying key considerations for the modeling problem and providing an empirical testbed for theories of how models can best learn from explanation data.

Subfields: Artificial Intelligence, Materials Chemistry, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Natural Language Processing

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, background): “As examples, it has been shown that natural language intermediate steps can improve performance (Zaidan et al., 2007; Yao et al., 2021; Hase and Bansal, 2022; Gu et al., 2022), improve robustness (Chen et al.”
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, neutral): “Similar to ours, recent works study the impact of language instructions in modifying a model behavior (Hase and Bansal, 2021; Ye and Ren, 2021; Gupta et al., 2021; Zhong et al., 2021).”
Training Language Models with Language Feedback (2022, methodology): “Prior work explores ways of using explanations to train text classiﬁcation models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Can language models learn from explanations in context? (2022, methodology): “…explored training or tuning of language processing models with task instructions (e.g. Raffel et al., 2020; Wei et al., 2021; Ouyang et al., 2022; Sanh et al., 2021), and with explanations (Cam-buru et al., 2018; Rajani et al., 2019; Narang et al., 2020; Zhou et al., 2020; Hase and Bansal, 2021).”
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, methodology): “But as with retrieval-based methods, this approach does not change the model weights and hence does not influence model outputs on all other potentially relevant inputs (Lewis et al., 2020; Hase and Bansal, 2021).”
FLEX: Unifying Evaluation for Few-Shot NLP (2021, background): “Also, given the importance of zero-shot learning and learning from task descriptions [28, 70], the benchmark needs to include zero-shot episodes and textual labels to enable measuring progress for models that do not use conventional supervised training, including methods that leverage the latent kno”
Improving Code Generation by Training with Natural Language Feedback (2023, background): “…aligns with numerous other works that have explored supervision via natural language in other ways, such as via explanations (Camburu et al., 2018; Hase & Bansal, 2021; Pruthi et al., 2021; Lampinen et al., 2022, inter alia ) and as part of RL systems (Fidler et al., 2017; Luketina et al., 2019;…”
Learning from Natural Language Feedback (2024, methodology): “Prior work explores ways of using explanations to train text classification models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021, inter alia ).”
Training Language Models with Language Feedback at Scale (2023, methodology): “Prior work explores ways of using explanations to train text classification models, with mixed results (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase & Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Tell me why! Explanations support learning relational and causal structure (2021, background): “Natural Language Processing: Explanations ﬁt naturally into NLP tasks, and Hase & Bansal (2021) highlight the many ways that explanations could enter in NLP tasks, e.g. as targets, inputs, or as priors.”
In-BoXBART: Get Instructions into Biomedical Multi-Task Learning (2022, background): “Studies have been made to investigate the effect of nat-ural language instructions on model performance (Hase and Bansal, 2021; Ye and Ren, 2021b; Zhong et al., 2021; Weller et al., 2020).”
Supervising Model Attention with Human Explanations for Robust Natural Language Inference (2021, methodology): “…(2020) also investigate training with explanations for sentiment analysis and question answering tasks, whereas we train with explanations for NLI, a task where most prior work ﬁnds that explanations do not improve performance (Hase and Bansal 2021; Kumar and Talukdar 2020; Cam-buru et al. 2018).”
Tell me why! - Explanations support learning of relational and causal structure (2021, background): “Natural Language Processing: Explanations ﬁt naturally into NLP tasks, and Hase and Bansal (2021) highlight the many ways that explanations could enter in NLP tasks, e.g. as targets, inputs, or as priors.”
Training Language Models with Natural Language Feedback (2022, methodology): “Prior work explores ways of using explanations to train text classiﬁcation models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language Instructions (2021, background): “A recent line of work has explored the ability of models to reason with respect to natural language description of a goal task (Goldwasser and Roth, 2014; Weller et al., 2020; Efrat and Levy, 2020; Clark et al., 2020; Hase and Bansal, 2021; Ye and Ren, 2021).”
Natural Language Inference with a Human Touch: Using Human Explanations to Guide Model Attention (2021, methodology): “2) We show that when combined with DeBERTa (He et al., 2021), this approach achieves a new state-of-the-art result for SNLI.”
How Many Data Samples is an Additional Instruction Worth? (2022, background): “After that, many works have been proposed to show that models follow language instructions (Hase and Bansal, 2021; Ye and Ren, 2021; Gupta et al., 2021; Zhong et al., 2021).”