R3: Reinforced Ranker-Reader for Open-Domain Question Answering
By Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Shiyu Chang, G. Tesauro, Bowen Zhou, Jing Jiang. 2018
200 citations recorded by the source corpus.
In recent years researchers have achieved considerable success applying neural network methods to question answering (QA). These approaches have achieved state of the art results in simplified closed-domain settings such as the SQuAD (Rajpurkar et al. 2016) dataset, which provides a pre-selected passage, from which the answer to a given question may be extracted. More recently, researchers have begun to tackle open-domain QA, in which the model is given a question and access to a large corpus (e.g., wikipedia) instead of a pre-selected passage (Chen et al. 2017a). This setting is more complex as it requires large-scale search for relevant passages by an information retrieval component, combined with a reading comprehension model that “reads” the passages to generate an answer to the question. Performance in this setting lags well behind closed-domain performance. In this paper, we present a novel open-domain QA system called Reinforced Ranker-Reader (R3), based on two algorithmic innovations. First, we propose a new pipeline for open-domain QA with a Ranker component, which learns to rank retrieved passages in terms of likelihood of extracting the ground-truth answer to a given question. Second, we propose a novel method that jointly trains the Ranker along with an answer-extraction Reader model, based on reinforcement learning. We report extensive experimental results showing that our method significantly improves on the state of the art for multiple open-domain QA datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Information Retrieval & Search
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Jianfeng Gao
- Percy Liang
- Caiming Xiong
- Graham Neubig
- Yejin Choi
In Computer Vision and Pattern Recognition
- Hannaneh Hajishirzi
- Mohit Bansal
- Luc Van Gool
- Jianfeng Gao
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Cordelia Schmid
In Retrieval-Augmented Generation
- Hannaneh Hajishirzi
- Sebastian Riedel
- Akari Asai
- Danqi Chen
- Wen-tau Yih
- Fabio Petroni
- Gautier Izacard
- Patrick Lewis
In Information Retrieval & Search
- Hannaneh Hajishirzi
- Sebastian Riedel
- Wen-tau Yih
- Danqi Chen
- Gautier Izacard
- Akari Asai
- Edouard Grave
- Fabio Petroni
Citations in the corpus
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, methodology): “Some work optimizes the retrieval module to aid in a specific, downstream task such as question answering, using search [40], reinforcement learning [6, 57, 56], or a latent variable approach [27, 18] as in our work.”
- Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “This approach has been shown to work well previously (Wang et al., 2019, 2018; Lin et al., 2018).”
- Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, methodology): “Lee et al. (2018) introduced a supervised learning method to rerank paragraphs based on BiLSTM, while Wang et al. (2018a) trained a ranking system with reinforcement learning.”
- How Context Affects Language Models' Factual Predictions (2020, methodology): “Works in this areas mostly focus on enhancing retrieval quality [Choi et al., 2017, Wang et al., 2018b,c, Lin et al., 2018, Min et al., 2018, Lee et al., 2018, 2019a, Das et al., 2019, Xiong et al., 2019], improving answer aggregation [Clark and Gardner, 2017, Wang et al., 2018c, Lee et al., 2018,…”
- Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “…this formulation, a couple of recent studies have proposed to improve the system us-ing stronger reading comprehension models (Yang et al., 2019; Wang et al., 2018), more effective learning objectives (Clark and Gardner, 2018; Min et al., 2019a; Wang et al., 2019) or paragraph reranking models…”
- Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “Although using a single dump for different open-domain QA datasets is a common practice (Chen et al., 2017; Wang et al., 2018a; Lee et al., 2018), this potentially causes inconsistent or even unfair evaluation across different experimental settings, due to the temporal inconsistency of the…”
- Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “Recent work has focused on two stage approaches that combine information retrieval with neural reading compression (Chen et al., 2017; Wang et al., 2018; Das et al., 2019; Yang et al., 2019).”
- Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, background): “Machine Reading at Scale First proposed and formalized in Chen et al. (2017), MRS has gained popularity with increasing amount of work on both dataset collection (Joshi et al., 2017; Welbl et al., 2018) and MRS model developments (Wang et al., 2018; Clark and Gardner, 2017; Htut et al., 2018).”
- Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering (2019, methodology): “In future, we plan to consider inter-correlation among passages for open-domain question answering (Wang et al., 2018b; Song et al., 2018).”
- Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, methodology): “Open-domain QA systems such as R 3 (Wang et al., 2018a) and DS-QA (Lin et al., 2018) have sophisticated retriever models where the reader and retriever are jointly trained.”
- Denoising Distantly Supervised Open-Domain Question Answering (2018, methodology): “Hence, Wang et al. (2018b) propose strength-base and coverage-based re-ranking approaches, which can aggregate the results extracted from each paragraph by existing DS-QA system to better determine the answer.”
- R2-D2: A Modular Baseline for Open-Domain Question Answering (2021, background): “Previous work in QA based on neural nets used Bi-LSTM encoders (Wang et al., 2018; Lee et al., 2018) that score each document independently.”
- End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering (2021, background): “There are some similarities between EMDR2 and Lalt-2 to Hard EM (Min et al., 2019) and Reinforced Reader-Ranker (R3; Wang et al. (2018)), at the conceptual level even though they are not equivalent.”
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, methodology): “(Wang et al., 2018) proposes making use of the aggregated evidence from across multiple paragraphs to better determine the answer with re-rankers.”
- Cognitive Graph for Multi-Hop Reading Comprehension at Scale (2019, background): “Improvements are made to enhance retrieval by heuristic sampling (Clark and Gardner, 2018) or reinforcement learning (Hu et al., 2018; Wang et al., 2018a), while for complex reasoning, necessary revisits to the framework are neglected.”
- Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model (2019, methodology): “QA model with the following systems: DrQA (Chen et al., 2017) is an open-domain QA system which uses TF-IDF with bigram features for ranking and a simple attentive reader for answer extraction. R3 (Wang et al., 2018a) is a reinforcement learning based system which jointly trains a paragraph ranker a”
- Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (2018, methodology): “The question-answer pairs were generated by annotators from a paragraph; and although the dataset is typically used to evaluate reading comprehension, it has also been used in an open domain QA setting (Chen et al., 2017; Wang et al., 2018).”
- Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, background): “Finally, various approaches (Chen et al., 2016; Buck et al., 2018; Nogueira and Cho, 2017; Wang et al., 2018) have also proposed reformulating questions for textual QA tasks.”