Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model

By Jacob Eisenstein, D. Andor, Bernd Bohnet, Michael Collins, David M. Mimno. 2022

27 citations recorded by the source corpus.

Explainable question answering systems should produce not only accurate answers but also rationales that justify their reasoning and allow humans to check their work. But what sorts of rationales are useful and how can we train systems to produce them? We propose a new style of rationale for open-book question answering, called \emph{markup-and-mask}, which combines aspects of extractive and free-text explanations. In the markup phase, the passage is augmented with free-text markup that enables each sentence to stand on its own outside the discourse context. In the masking phase, a sub-span of the marked-up passage is selected. To train a system to produce markup-and-mask rationales without annotations, we leverage in-context learning. Specifically, we generate silver annotated data by sending a series of prompts to a frozen pretrained language model, which acts as a teacher. We then fine-tune a smaller student model by training on the subset of rationales that led to correct answers. The student is"honest"in the sense that it is a pipeline: the rationale acts as a bottleneck between the passage and the answer, while the"untrusted"teacher operates under no such constraints. Thus, we offer a new way to build trustworthy pipeline systems from a combination of end-task annotations and frozen pretrained language models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Safety, Robustness & Interpretability, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Safety, Robustness & Interpretability

In Question Answering & Reading Comprehension

Citations in the corpus

Teaching Small Language Models to Reason (2022, background): “Yet differently, the focus of Eisenstein et al. (2022) relies on producing markup-and-mask explanations for open-book question answering.”
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes (2023, methodology): “…to the input examples and train the model to output differently based on the given task preﬁx (Section 3.2). man rationales can be used as gold standard labels to make models more interpretable by generating similar rationales (Wiegreffe et al., 2021; Narang et al., 2020; Eisenstein et al., 2022).”
Distilling Reasoning Capabilities into Smaller Language Models (2022, result): “Similar to our hypothesis, Eisenstein et al. (2022) argued that question-answering systems should focus not only on the ﬁnal answer, but also on the rationale that justiﬁes their reasoning, to help them reason better.”
Distilling Multi-Step Reasoning Capabilities of Large Language Models into Smaller Models via Semantic Decompositions (2022, result): “Similar to our hypothesis, Eisenstein et al. (2022) argued that question-answering systems should not only focus on the ﬁnal answer but also rationales that justify their reasoning to help reason better.”