Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
By Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Andrew McCallum. 2019
173 citations recorded by the source corpus.
This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other. The framework is agnostic to the architecture of the machine reading model, only requiring access to the token-level hidden representations of the reader. The retriever uses fast nearest neighbor search to scale to corpora containing millions of paragraphs. A gated recurrent unit updates the query at each step conditioned on the state of the reader and the reformulated query is used to re-rank the paragraphs by the retriever. We conduct analysis and show that iterative interaction helps in retrieving informative paragraphs from the corpus. Finally, we show that our multi-step-reasoning framework brings consistent improvement when applied to two widely used reader architectures DrQA and BiDAF on various large open-domain datasets --- TriviaQA-unfiltered, QuasarT, SearchQA, and SQuAD-Open.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Information Retrieval & Search
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Hannaneh Hajishirzi
- Caiming Xiong
- Graham Neubig
- Yejin Choi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jianfeng Gao
- Dacheng Tao
- Mohit Bansal
- Trevor Darrell
- Hannaneh Hajishirzi
- Xiaogang Wang
- Andrew Zisserman
In Retrieval-Augmented Generation
- Hannaneh Hajishirzi
- Sebastian Riedel
- Danqi Chen
- Wen-tau Yih
- Akari Asai
- Matei A. Zaharia
- Christopher Potts
- Omar Khattab
In Information Retrieval & Search
- Wen-tau Yih
- Hannaneh Hajishirzi
- Sebastian Riedel
- Danqi Chen
- Jinhyuk Lee
- Patrick Lewis
- Jimmy J. Lin
- Akari Asai
Citations in the corpus
- How Context Affects Language Models' Factual Predictions (2020, background): “…mostly focus on enhancing retrieval quality [Choi et al., 2017, Wang et al., 2018b,c, Lin et al., 2018, Min et al., 2018, Lee et al., 2018, 2019, Das et al., 2019, Xiong et al., 2019], improving answer aggregation [Clark and Gardner, 2018, Wang et al., 2018c, Lee et al., 2018, Pang et al.,…”
- Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, background): “While recent work on improving evidence retrieval has made significant progress (Wang et al., 2018; Kratzwald and Feuer-riegel, 2018; Lee et al., 2018; Das et al., 2019), they still only rerank a closed evidence set.”
- Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “…we use the following metrics: Answer Recall (AR), which evaluates the recall of the answer string among top paragraphs (Wang et al., 2018a; Das et al., 2019), Paragraph Recall (PR), which evaluates if at least one of the ground-truth paragraphs is included among the retrieved paragraphs,…”
- Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “Recent work has focused on two stage approaches that combine information retrieval with neural reading compression (Chen et al., 2017; Wang et al., 2018; Das et al., 2019; Yang et al., 2019).”
- Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “…phrase-level question answering has gained a great popularity (Shen et al., 2017; Raiman and Miller, 2017; Min et al., 2018; Raison et al., 2018; Das et al., 2019), where a few (5-10) documents relevant to the question are retrieved and then a deep neural model finds the answer in the document.”
- Adaptive Information Seeking for Open-Domain Question Answering (2021, methodology): “For example, some works employ a single retrieval function multiple times (Das et al., 2019a; Qi et al., 2019; Xiong et al., 2021), and the other works use a pre-defined sequence of retrieval functions (Asai et al., 2020; Dhingra et al., 2020).”
- MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension (2019, neutral): “…although it does not even 1 State-of-the-are-results were found in (Tay et al., 2018) for N EWS QA, in Lin et al. (2018), for S EARCH QA, in Das et al. (2019) for TQA-U, in (Talmor and Berant, 2018b) for CWQ, in Ding et al. (2019) for H OTPOT QA, in (Abujabal et al., 2018) for C OM QA,…”
- Pre-training Tasks for Embedding-based Large-scale Retrieval (2020, neutral): “Das et al. (2019) encode query and document separately with LSTM encoders.”
- Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “Prior to these methods based on pretrained models, Das et al. (2019) use RNN encoder to get dense representations of questions and passages.”
- Differentiable Reasoning over a Virtual Knowledge Base (2020, methodology): “The same limitation is shared by methods which perform multi-step retrieval interleaved with a reading comprehension model (Das et al., 2019a; Feldman & El-Yaniv, 2019; Lee et al., 2019).”
- Multi-hop Question Answering via Reasoning Chains (2019, background): “A recent line of works on open-domain multi-hop QA (Feldman and El-Yaniv, 2019; Das et al., 2019; Asai et al., 2019; Qi et al., 2019; Godbole et al., 2019) also adopt the idea of forming query in a iterative way to select the most relevant documents regarding the question.”
- Boosting Search Engines with Interactive Agents (2021, neutral): “Das et al. (2019) propose to perform query reformulation in embedding (continuous) space and find that it can outperform the sequence-based approach.”
- Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering (2019, neutral): “A few recent works learn to reformulate the query using task specific reward such as document recall or performance on a QA task (Nogueira and Cho, 2017; Buck et al., 2018; Das et al., 2019).”
- What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams (2020, background): “After-wards, researchers have introduced more sophisticated models, which either aggregate all informative evidence (Clark and Gardner 2018; Wang et al. 2019) or filter out those irrelevant retrieved texts (Wang et al. 2018; Das et al. 2019) to better predict the answers.”
- Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering (2018, methodology): “We first introduce the reader model in Section 3.1 and then the essential term selector in Section 3.2.”
- Contextualized Sparse Representation with Rectified N-Gram Attention for Open-Domain Question Answering (2019, methodology): “…al., 2017) 29.8 ** - 25.4 * 35 R 3 (Wang et al., 2018a) 29.1 37.5 28.4 * - Paragraph Ranker (Lee et al., 2018) 30.2 - 35.4 * - Multi-Step-Reasoner (Das et al., 2019) 31.9 39.2 - - BERTserini (Yang et al., 2019) 38.6 46.1 - 115 ORQA (Lee et al., 2019) 20.2 - 30.1 - Multi-passage BERT †† (Wang et…”
- Answering Complex Open-domain Questions Through Iterative Query Generation (2019, methodology): “Das et al. (2019) propose a neural retriever trained with distant supervision to bias towards paragraphs containing answers to the given questions, which is then used in a multi-step reader-reasoner framework.”
- Simple and Efficient ways to Improve REALM (2021, methodology): “Recently, dense retrieval methods (Seo et al., 2018, 2019; Das et al., 2019; Karpukhin et al., 2020) have improved over sparse retrievers like BM25 (Robertson and Zaragoza, 2009) and made training these systems end-to-end by leveraging approximate MIPS search (Shrivastava and Li, 2014).”