Finding Generalizable Evidence by Learning to Convince Q&A Models

By Ethan Perez, Siddharth Karamcheti, Rob Fergus, Jason Aaron Edward Weston, Douwe Kiela, Kyunghyun Cho. 2019

40 citations recorded by the source corpus.

We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed. We train evidence agents to select the passage sentences that most convince a pretrained QA model of a given answer, if the QA model received those sentences instead of the full passage. Rather than finding evidence that convinces one model alone, we find that agents select evidence that generalizes; agent-chosen evidence increases the plausibility of the supported answer, as judged by other QA models and humans. Given its general nature, this approach improves QA in a robust manner: using agent-selected evidence (i) humans can correctly answer questions with only ~20% of the full passage and (ii) QA models can generalize to longer passages and harder questions.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, methodology): “Some work optimizes the retrieval module to aid in a speciﬁc, downstream task such as question answering, using search [40], reinforcement learning [6, 57, 56], or a latent variable approach [27, 18] as in our work.”
Training language models to follow instructions with human feedback (2022, background): “…dialogue (Jaques et al., 2019; Yi et al., 2019; Hancock et al., 2019), translation (Kreutzer et al., 2018; Bahdanau et al., 2016), semantic parsing (Lawrence and Riezler, 2018), story generation (Zhou and Xu, 2020), review generation (Cho et al., 2018), and evidence extraction (Perez et al., 2019).”
Fine-Tuning Language Models from Human Preferences (2019, background): “Although language models are unlikely to be ready for these tasks in their full generality, Perez et al. (2019) demonstrates that debate already improves generalization for question-answering when debaters quote from a source text.”
Learning to summarize from human feedback (2020, background): “Human feedback has also been used as a reward to train models in other domains such as dialogue [25, 68, 21], translation [32, 1], semantic parsing [34], story generation [72], review generation [7], and evidence extraction [46].”
Recursively Summarizing Books with Human Feedback (2021, background): “…et al., 2018; Bahdanau et al., 2016), semantic parsing (Lawrence and Riezler, 2018), story generation (Zhou and Xu, 2020), review generation (Cho et al., 2018), and evidence extraction (Perez et al., 2019), and agents in simulated environments (Christiano et al., 2017; Ibarz et al., 2018).”
Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “Initial work towards multistep human interaction methods includes simulated debate using frozen question answering models (Perez et al., 2019) and recursively summarising books (Wu et al., 2021), which simplifies the rating task from evaluating book-length summaries to passagelength summaries.”
Self-critiquing models for assisting human evaluators (2022, background): “[PKF19] help humans perform passage-based question-answering, without reading much of the passages.”
Red Teaming Language Models with Language Models (2022, methodology): “Adversarial Transfer: Adversarial inputs often transfer across models (Szegedy et al., 2014; Liu et al., 2017; Perez et al., 2019), in which case it is easy for adversaries to attack a new model if they have attacked others.”
Teaching language models to support answers with verified quotes (2022, methodology): “For brevity, we limited baselines drawn from the question-answering literature to just this model, though it may also be worthwhile to consider comparing to a less standard objective such as that of Perez et al. (2019) in future work.”
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length (2021, methodology): “Such reasons include written explanations (Camburu et al., 2018; Rajani et al., 2019; Wiegreffe et al., 2020) or locating taskrelevant input words (Zhang et al., 2016b; Perez et al., 2019).”
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020, methodology): “In the context of QA, there are multiple notions of explanation/justiﬁcation, including showing an authoritative, answer-bearing sentence (Perez et al., 2019), a collection of text snippets supporting an answer (DeYoung et al., 2020), an attention map over a passage (Seo et al., 2016), a…”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “For example, one might design the AI to summarize the best evidence for all possible options, without giving hard predictions, by training evidence agents [64].”
Frustratingly Hard Evidence Retrieval for QA Over Books (2020, background): “Another possible direction is to extend the idea of inferring evidence on training data with game-theoretic approaches (Perez et al., 2019; Feng et al., 2020), then use the inferred evidence paragraph as labels to train the ranker.”
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development (2023, methodology): “Thus, RLHF has been positively applied to fine-tuning language models in most domains in natural language process (NLP) such as dialogue [88], [89], translation [90], [91], story generation [92], evidence extraction [93], [94], and semantic parsing [95].”
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021, methodology): “This emerging interpretability is considerably different from methods that explicitly train models to output explanations (Perez et al., 2019; Schuff et al., 2020).”