Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering

By Jinhyuk Lee, Seongjun Yun, Hyunjae Kim, Miyoung Ko, Jaewoo Kang. 2018

90 citations recorded by the source corpus.

Recently, open-domain question answering (QA) has been combined with machine comprehension models to find answers in a large knowledge source. As open-domain QA requires retrieving relevant documents from text corpora to answer questions, its performance largely depends on the performance of document retrievers. However, since traditional information retrieval systems are not effective in obtaining documents with a high probability of containing answers, they lower the performance of QA systems. Simply extracting more documents increases the number of irrelevant documents, which also degrades the performance of QA systems. In this paper, we introduce Paragraph Ranker which ranks paragraphs of retrieved documents for a higher answer recall with less noise. We show that ranking paragraphs and aggregating answers using Paragraph Ranker improves performance of open-domain QA pipeline on the four open-domain QA datasets by 7.8% on average.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, methodology): “Lee et al. (2018) introduced a supervised learning method to rerank paragraphs based on BiLSTM, while Wang et al. (2018a) trained a ranking system with reinforcement learning.”
Improving Question Answering with External Knowledge (2019, methodology): “trieving relevant documents in Wikipedia and then running a MRC model to extract a text span from the documents based on the question (Chen et al., 2017; Wang et al., 2018b; Kratzwald and Feuerriegel, 2018; Lee et al., 2018; Lin et al., 2018).”
Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, background): “While recent work on improving evidence retrieval has made significant progress (Wang et al., 2018; Kratzwald and Feuerriegel, 2018; Lee et al., 2018; Das et al., 2019), they still only rerank a closed evidence set.”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “…to improve the system us-ing stronger reading comprehension models (Yang et al., 2019; Wang et al., 2018), more effective learning objectives (Clark and Gardner, 2018; Min et al., 2019a; Wang et al., 2019) or paragraph reranking models (Wang et al., 2017; Lin et al., 2018; Lee et al., 2018).”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “Although using a single dump for different open-domain QA datasets is a common practice (Chen et al., 2017; Wang et al., 2018a; Lee et al., 2018), this potentially causes inconsistent or even unfair evaluation across different experimental settings, due to the temporal inconsistency of the…”
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, methodology): “In some previous work (Lee et al., 2018), paragraph-level retrieval modules were mainly for improving the recall of required information, while in some other works (Yang et al., 2018), sentence-level retrieval modules were merely for solving the auxiliary sentence selection task.”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, neutral): “(Lee et al., 2018) 28.”
End-to-End Open-Domain Question Answering with BERTserini (2019, background): “Other recent papers have examined the role of retrieval in this end-to-end formulation (Wang et al., 2017; Kratzwald and Feuerriegel, 2018; Lee et al., 2018), some of which have, in essence, rediscovered ideas from the late 1990s and early 2000s.”
R2-D2: A Modular Baseline for Open-Domain Question Answering (2021, background): “Previous work in QA based on neural nets used Bi-LSTM encoders (Wang et al., 2018; Lee et al., 2018) that score each document independently.”
Adaptive Information Seeking for Open-Domain Question Answering (2021, methodology): “In the retrieval stage, most approaches employ a de-terminate retrieval function and treat each passage independently (Wang et al., 2018; Lin et al., 2018; Lee et al., 2018; Yang et al., 2018; Pang et al., 2019; Lee et al., 2019; Guu et al., 2020; Karpukhin et al., 2020; Izacard and Grave, 2021).”
Contextualized Sparse Representation with Rectified N-Gram Attention for Open-Domain Question Answering (2019, background): “, 2017) or weak supervision (Lee et al., 2018; Min et al., 2019) on CuratedTREC.”
Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering (2021, unknown): “Some previous work (Kratzwald and Feuer-riegel, 2018; Lee et al., 2018; Das et al., 2019; Wang et al., 2018) train the end-to-end models to rerank in a closed set.”
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2020, methodology): “To address this problem, prior work has focused on replacing the ﬁrst stage retriever with a train-1 able ranker (Chidambaram et al., 2018; Lee et al., 2018; Wang et al., 2018).”
Contextualized Sparse Representations for Real-Time Open-Domain Question Answering (2019, methodology): “Comparisons For models using dedicated search engines, we show performances of DrQA (Chen et al., 2017), R 3 (Wang et al., 2018), Para-graph Ranker (Lee et al., 2018), Multi-Step-Reasoner (Das et al., 2019), BERTserini (Yang et al., 2019), and Multi-passage BERT (Wang et al., 2019).”
Reader-Guided Passage Reranking for Open-Domain Question Answering (2021, methodology): “, 2019) and explored in early OpenQA systems (Wang et al., 2018a; Lee et al., 2018).”
Open-Retrieval Conversational Question Answering (2020, background): “Some previous work [17, 22, 25, 48] learns to rerank or select from a closed set of passages for open domain QA.”
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020, background): “Later, other proposals were able to surpass its quality of answer with a paragraph reranker (Lee et al. 2018; Wang et al. 2017) and then with a ”minimal” retriever that selects small but relevant portions of text (Min et al. 2018).”