Adaptive Document Retrieval for Deep Question Answering

By Bernhard Kratzwald, Stefan Feuerriegel. 2018

56 citations recorded by the source corpus.

State-of-the-art systems in deep question answering proceed as follows: (1)an initial document retrieval selects relevant documents, which (2) are then processed by a neural network in order to extract the final answer. Yet the exact interplay between both components is poorly understood, especially concerning the number of candidate documents that should be retrieved. We show that choosing a static number of documents - as used in prior research - suffers from a noise-information trade-off and yields suboptimal results. As a remedy, we propose an adaptive document retrieval model. This learns the optimal candidate number for document retrieval, conditional on the size of the corpus and the query. We report extensive experimental results showing that our adaptive approach outperforms state-of-the-art methods on multiple benchmark datasets, as well as in the context of corpora with variable sizes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Improving Question Answering with External Knowledge (2019, methodology): “trieving relevant documents in Wikipedia and then running a MRC model to extract a text span from the documents based on the question (Chen et al., 2017; Wang et al., 2018b; Kratzwald and Feuerriegel, 2018; Lee et al., 2018; Lin et al., 2018).”
Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, background): “While recent work on improving evidence retrieval has made significant progress (Wang et al., 2018; Kratzwald and Feuerriegel, 2018; Lee et al., 2018; Das et al., 2019), they still only rerank a closed evidence set.”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, background): “On the other hand, increasing the candidate paragraphs introduces additional noises, may distract models, and eventually hurt the performance (Kratzwald & Feuerriegel, 2018).”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, neutral): “5 Kratzwald and Feuerriegel (2018) 29.”
End-to-End Open-Domain Question Answering with BERTserini (2019, background): “Other recent papers have examined the role of retrieval in this end-to-end formulation (Wang et al., 2017; Kratzwald and Feuerriegel, 2018; Lee et al., 2018), some of which have, in essence, rediscovered ideas from the late 1990s and early 2000s.”
Combining Fact Extraction and Verification with Neural Semantic Matching Networks (2018, background): “ir work, Dhingra, Mazaitis, and Cohen (2017) introduced new benchmarks, Wang et al. (2018) extended the framework by adding feedback signals from comprehension to the upstream document retriever, and Kratzwald and Feuerriegel (2018) proposed to adaptively adjust the number of retrieved documents. FE”
Learning from On-Line User Feedback in Neural Question Answering on the Web (2019, methodology): “by our implementation, in which we combine several of the recent advances from neural question answering, such as attention [40, 43] and adaptive retrieval [23], in order to obtain a reference system that represents the status quo.”
ReQA: An Evaluation for End-to-End Answer Retrieval Models (2019, background): “Successful systems usually follow a two-step approach to answer a given question: first retrieve relevant articles or blocks, and then scan the returned text to identify the answer using a reading comprehension model (Jurafsky and Martin, 2018; Kratzwald and Feuerriegel, 2018; Yang et al., 2019a; Le”
Open-Retrieval Conversational Question Answering (2020, background): “Some previous work [17, 22, 25, 48] learns to rerank or select from a closed set of passages for open domain QA.”
Document Re-Ranking Model for Machine-Reading and Comprehension (2020, background): “Therefore, cert models ave b en proposed for enhancing IR erformance [6].”
Answering Complex Open-domain Questions Through Iterative Query Generation (2019, methodology): “Recent work on open-domain question answering largely follow this retrieve-and-read approach, and focus on improving the information retrieval component with question answering performance in consideration (Nishida et al., 2018; Kratzwald and Feuerriegel, 2018; Nogueira et al., 2019).”
Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering (2020, background): “One line of work jointly learns RC and evidence ranking using either a pipeline system (Wang et al., 2018a; Lee et al., 2018; Kratzwald and Feuerriegel, 2018) or an end-to-end model (Lee et al., 2019).”
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, methodology): “Their experiments demonstrate that pre-trained language models are able to gain impressive performance on various benchmarks and such Retrievalfree methods make a fundamentally different approach to building OpenQA systems.”