Coarse-to-Fine Question Answering for Long Documents

By Eunsol Choi, Daniel Hewlett, Jakob Uszkoreit, Illia Polosukhin, Alexandre Lacoste, Jonathan Berant. 2016

170 citations recorded by the source corpus.

We present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models. While most successful approaches for reading comprehension rely on recurrent neural networks (RNNs), running them over long documents is prohibitively slow because it is difficult to parallelize over sequences. Inspired by how people first skim the document, identify relevant parts, and carefully read these parts to produce an answer, we combine a coarse, fast model for selecting relevant sentences and a more expensive RNN for producing the answer from those sentences. We treat sentence selection as a latent variable trained jointly from the answer only using reinforcement learning. Experiments demonstrate state-of-the-art performance on a challenging subset of the WikiReading dataset and on a new dataset, while speeding up the model by 3.5x-6.7x.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, methodology): “Some work optimizes the retrieval module to aid in a speciﬁc, downstream task such as question answering, using search [40], reinforcement learning [6, 57, 56], or a latent variable approach [27, 18] as in our work.”
Parameter-Efficient Transfer Learning for NLP (2019, background): “…examples include: text processing systems (part of speech, chunking, named entity recognition, etc.) (Collobert & Weston, 2008), multilingual models (Huang et al., 2013), semantic parsing (Peng et al., 2017), machine translation (Johnson et al., 2017), and question answering (Choi et al., 2017).”
Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, background): “The task of selecting sentences to answer the question has been studied across several QA datasets (Min et al., 2018b; Wang et al., 2019; Choi et al., 2017).”
How Context Affects Language Models' Factual Predictions (2020, background): “Works in this areas mostly focus on enhancing retrieval quality [Choi et al., 2017, Wang et al., 2018b,c, Lin et al., 2018, Min et al., 2018, Lee et al., 2018, 2019, Das et al., 2019, Xiong et al., 2019], improving answer aggregation [Clark and Gardner, 2018, Wang et al., 2018c, Lee et al., 2018,…”
Finding Generalizable Evidence by Learning to Convince Q&A Models (2019, background): “Others extract evidence to improve downstream QA efficiency over large amounts of text (Choi et al., 2017; Kratzwald and Feuerriegel, 2019; Wang et al., 2018b).”
Denoising Distantly Supervised Open-Domain Question Answering (2018, methodology): “Our work is also inspired by the idea of coarse-to-ﬁne models in NLP. Cheng and Lapata (2016) and Choi et al. (2017) propose a coarse-to-ﬁne model, which ﬁrst selects essential sentences and then performs text summarization or reading comprehension on the chosen sentences respectively.”
Efficient and Robust Question Answering from Minimal Context over Documents (2018, background): “Choi et al. (2017) propose a framework that identiﬁes the sentences relevant to the question (property) using simple bag-of-words representation, then generates the answer from those sentences using recurrent neural networks.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “In question answering (QA), Choi et al. (2017) extract one sentence and then generate the answer from the sentence’s vector representation with RL bridging.”
R3: Reinforced Reader-Ranker for Open-Domain Question Answering (2017, positive): “(Cheng and Lapata, 2016) and (Choi et al., 2017) proposed coarse-to-ﬁne frameworks with an additional sentence-level prediction followed by the original word-level prediction for text summarization and reading comprehension, respectively.”
Multi-Mention Learning for Reading Comprehension with Neural Cascades (2017, background): “Approaches such as Choi et al. (2017) ﬁrst select the top (1-2) sentences using a very coarse model and then run a recurrent architecture on these sentences to ﬁnd the correct span.”
Hierarchical Graph Network for Multi-hop Question Answering (2019, background): “Choi et al. (2017) proposes to use reinforcement learning to first select relevant sentences and then produce answers from those sentences.”
Multi-hop Question Answering via Reasoning Chains (2019, methodology): “Our approach resembles past models for coarse-to-fine question answering (Choi et al., 2017; Min et al., 2018; Wang et al., 2019), but explores the context in a sequential fashion and is trained to produce principled chains.”
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (2019, background): “Choi et al. (2017); Zhong et al. (2019) proposed a combination of coarse reading and ﬁne reading.”
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension (2018, background): “Hence other recent approaches propose to jointly learn them via reinforcement learning (Choi et al., 2017; Wang et al., 2018; Nogueira and Cho, 2017) or cascades (Swayamdipta et al.”
Evidence Sentence Extraction for Machine Reading Comprehension (2019, background): “We are aware of some similar work (Choi et al., 2017; Lin et al., 2018; Htut et al., 2018) that aim to select relevant paragraphs for question answering tasks.”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, methodology): “The second group of methods tend to rely on reinforcement learning (Choi et al., 2017; Lai et al., 2018; Geva and Berant, 2018) or PageRank (Surdeanu et al.”