Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension

By Kyosuke Nishida, Itsumi Saito, Atsushi Otsuka, Hisako Asano, Junji Tomita. 2018

53 citations recorded by the source corpus.

This study considers the task of machine reading at scale (MRS) wherein, given a question, a system first performs the information retrieval (IR) task of finding relevant passages in a knowledge source and then carries out the reading comprehension (RC) task of extracting an answer span from the passages. Previous MRS studies, in which the IR component was trained without considering answer spans, struggled to accurately find a small number of relevant passages from a large set of passages. In this paper, we propose a simple and effective approach that incorporates the IR and RC tasks by using supervised multi-task learning in order that the IR component can be trained by considering answer spans. Experimental results on the standard benchmark, answering SQuAD questions using the full Wikipedia as the knowledge source, showed that our model achieved state-of-the-art performance. Moreover, we thoroughly evaluated the individual contributions of our model components with our new Japanese dataset and SQuAD. The results showed significant improvements in the IR task and provided a new perspective on IR for RC: it is effective to teach which part of the passage answers the question rather than to give only a relevance score to the whole passage.

Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Retrieval-Augmented Generation

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “For instance, retrieved documents could be re-ranked using reinforcement learning (Wang et al., 2018a), distant supervision (Lin et al., 2018), or multi-task learning (Nishida et al., 2018).”
Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering (2018, methodology): “Nishida et al. (2018) developed a multi-task learning (MTL) method for a retriever and reader in order to obtain a strong retriever that considers certain passages including the answer text as positive samples during training.”
Effective Subword Segmentation for Text Comprehension (2018, background): “prehension, though the task formations are slightly different. In the last decade, the MRC tasks have evolved from the early cloze-style test [19, 20, 54] to span-based answer extraction from passage [36, 39, 40]. The former has restrictions that each answer should be a single word in the document a”
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, methodology): “For example, [15] propose to jointly train Retriever and Reader using multi-task learning based on the BiDAF model [23], simultaneously computing the similarity of a passage to the given question and predicting the start and end position of an answer span.”
Knowledge Graph Prompting for Multi-Document Question Answering (2023, methodology): “Although this framework has been broadly applied in O-QA (Mao et al. 2021; Nishida et al. 2018) and D-QA (Xu et al.”
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (2019, methodology): “For such datasets, the document retrieval model is combined with the contextquery matching model (Chen et al., 2017a; Wang et al., 2018a,b; Nishida et al., 2018).”
Answering Complex Open-domain Questions Through Iterative Query Generation (2019, methodology): “Recent work on open-domain question answering largely follow this retrieve-and-read approach, and focus on improving the information retrieval component with question answering performance in consideration (Nishida et al., 2018; Kratzwald and Feuer-riegel, 2018; Nogueira et al., 2019).”
Multi-style Generative Reading Comprehension (2019, methodology): “…find a small number of relevant passages with a TF-IDF based ranker and pass them to a neural reader (Chen et al., 2017; Clark and Gardner, 2018), while more recent pipeline models use a neural re-ranker to more accurately select the relevant passages (Wang et al., 2018; Nishida et al., 2018).”