R3: Reinforced Reader-Ranker for Open-Domain Question Answering

By Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, G. Tesauro, Bowen Zhou, Jing Jiang. 2017

64 citations recorded by the source corpus.

In recent years researchers have achieved considerable success applying neural network methods to question answering (QA). These approaches have achieved state of the art results in simplified closed-domain settings such as the SQuAD (Rajpurkar et al., 2016) dataset, which provides a pre-selected passage, from which the answer to a given question may be extracted. More recently, researchers have begun to tackle open-domain QA, in which the model is given a question and access to a large corpus (e.g., wikipedia) instead of a pre-selected passage (Chen et al., 2017a). This setting is more complex as it requires large-scale search for relevant passages by an information retrieval component, combined with a reading comprehension model that "reads" the passages to generate an answer to the question. Performance in this setting lags considerably behind closed-domain performance. In this paper, we present a novel open-domain QA system called Reinforced Ranker-Reader $(R^3)$, based on two algorithmic innovations. First, we propose a new pipeline for open-domain QA with a Ranker component, which learns to rank retrieved passages in terms of likelihood of generating the ground-truth answer to a given question. Second, we propose a novel method that jointly trains the Ranker along with an answer-generation Reader model, based on reinforcement learning. We report extensive experimental results showing that our method significantly improves on the state of the art for multiple open-domain QA datasets.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

Simple and Effective Multi-Paragraph Reading Comprehension (2017, methodology): “Self-attention has been used in several prior works (Cheng et al., 2016; Wang et al., 2017b; Pan et al., 2017).”
Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering (2017, methodology): “…(Seo et al., 2017), a RC model with bidirectional attention flow; AQA (Buck et al., 2017), a reinforced system learning to aggregate the answers generated by the re-written questions; R3 (Wang et al., 2017), a reinforced model making use of a ranker for selecting passages to train the RC model.”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “To the best Method Dataset NaturalQuestions-Open WebQuestions CuratedTREC DrQA (Chen et al., 2017) - 20.7 25.7 R 3 (Wang et al., 2017) - 17.1 28.4 DSQA (Lin et al., 2018) - 25.6 29.1 HardEM (Min et al., 2019a) 28.1 - - PathRetriever (Asai et al., 2019) 32.6 - - WKLM (Xiong et al., from methods that…”
End-to-End Open-Domain Question Answering with BERTserini (2019, background): “Other recent papers have examined the role of retrieval in this end-to-end formulation (Wang et al., 2017; Kratzwald and Feuerriegel, 2018; Lee et al., 2018), some of which have, in essence, rediscovered ideas from the late 1990s and early 2000s.”
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (2018, methodology): “However, most of the open-domain QA pipelines depend on traditional information retrieval systems which use TF-IDF rankings (Chen et al., 2017; Wang et al., 2017b).”
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, methodology): “…for CosmosQA, for each answer, the input token sequence is modiﬁed as “ < SEP > context BiDAF (Seo et al., 2016) 28.60 34.60 25.90 28.50 - AQA (Buck et al., 2018) 40.50 47.40 - - - Rˆ3 (Wang et al., 2017a) 49.00 55.30 35.30 41.70 - DSQA (Lin et al., 2018) 49.00 55.30 42.30 49.30 - Evidence Agg.”
TWEETQA: A Social Media Focused Question Answering Dataset (2019, methodology): “…genres of spans as potential answers, we try to match the question with all possible spans in the tweet context and choose the span with the highest BLEU-1 score as the ﬁnal answer, which follows the method and implementation 8 of answer span selection for open-domain QA (Wang et al., 2017).”
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (2018, methodology): “We tune the hyper-parameters according to the Model ROUGE-L BLEU-1 FastQA Ext (Weissenborn et al., 2017) 33.67 33.93 Prediction (Wang and Jiang, 2016) 37.33 40.72 ReasoNet (Shen et al., 2017) 38.81 39.86 R-Net (Wang et al., 2017c) 42.89 42.22 S-Net (Tan et al., 2017) 45 Two simple yet effective tech”
UvA-DARE (Digital Academic Repository) Learning to Transform, Combine, and Reason in Open-domain Question Answering (unknown): “However, most open-domain question answering methods focus on only one or a few candidate documents by filtering out the less relevant documents to avoid dealing with noisy information and operate over the selected set of documents to extract the answer [26, 34, 35].”
Evidence Extraction for Machine Reading Comprehension with Deep Probabilistic Logic (2018, background): “…TriviaQAEM F1 EM F1 EM F1 GA (Dhingra et al., 2017a) 26.4 26.4 BiDAF Seo et al. (2016) 25.9 28.5 28.6 34.6 AQA(Buck et al., 2017) 40.5 47.4 R3(Wang et al., 2017) 35.3 41.7 49.0 55.3 47.3 53.7 Lin et al. (2018): AVG 38.5 45.7 55.6 61.0 42.6 48.2 Lin et al. (2018): FULL 42.2 49.3 58.8 64.5…”
Retrieve, Rerank, Read, then Iterate: Answering Open-Domain Questions of Arbitrary Complexity from Text (2020, background): “hension and retrieval components, researchers have also found reranking search results (Wang et al., 2018a) or answer candidates (Wang et al.”
Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading (2018, methodology): “Recently, a large body of work have been proposed to tackle the SQuAD dataset only e.g. in (Dhingra et al., 2016; Wang et al., 2017b; Xiong et al., 2017; Liu et al., 2017a; Hu et al., 2017; Huang et al., 2017; Liu et al., 2017b).”
Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation (2018, methodology): “They use an answer re-ranking approach to reorder the answer candidates generated by a base Q&A model, R 3 (Wang et al., 2017a).”
Training a Ranking Function for Open-Domain Question Answering (2018, methodology): “To improve the performance of the overall pipeline, Wang et al. (2017) propose Reinforced Ranker-Reader (R3) model.”