Training a Ranking Function for Open-Domain Question Answering

By Phu Mon Htut, Samuel R. Bowman, Kyunghyun Cho. 2018

49 citations recorded by the source corpus.

In recent years, there have been amazing advances in deep learning methods for machine reading. In machine reading, the machine reader has to extract the answer from the given ground truth paragraph. Recently, the state-of-the-art machine reading models achieve human level performance in SQuAD which is a reading comprehension-style question answering (QA) task. The success of machine reading has inspired researchers to combine Information Retrieval with machine reading to tackle open-domain QA. However, these systems perform poorly compared to reading comprehension-style QA because it is difficult to retrieve the pieces of paragraphs that contain the answer to the question. In this study, we propose two neural network rankers that assign scores to different passages based on their likelihood of containing the answer to a given question. Additionally, we analyze the relative importance of semantic similarity and word level relevance matching in open-domain QA.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, background): “Machine Reading at Scale First proposed and formalized in Chen et al. (2017), MRS has gained popularity with increasing amount of work on both dataset collection (Joshi et al., 2017; Welbl et al., 2018) and MRS model developments (Wang et al., 2018; Clark and Gardner, 2017; Htut et al., 2018).”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, methodology): “The primary reason for this degradation in performance is due to the retriever’s failure to find the relevant paragraphs for the machine reading model (Htut et al., 2018).”
A dataset and baselines for sequential open-domain question answering (2018, background): “The framework of combining information retrieval and neural reading comprehension has been the basis of several systems for answering open-domain questions over unstructured text (Chen et al., 2017; Wang et al., 2018; Clark and Gardner, 2018; Htut et al., 2018).”
Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension (2019, background): “For example, as the training proceeds, a neural retriever is able to provide more relevant context than an IR engine (Htut et al., 2018).”
Open-Retrieval Conversational Question Answering (2020, background): “Some previous work [17, 22, 25, 48] learns to rerank or select from a closed set of passages for open domain QA.”
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering (2019, methodology): “In an attempt to improve the retrieval of the TF-IDF based component, many existing works have used Distant Supervision (DS) to further re-rank the retrieved paragraphs (Htut et al., 2018; Yan et al., 2018).”
Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering (2019, methodology): “The typical pipeline of open-domain QA systems (Chen et al., 2017; Wang et al., 2018; Htut et al., 2018) is to ﬁrst use an IR system to retrieve a compact set of paragraphs and then run a machine reading model over the concatenated or reranked paragraphs.”
UvA-DARE (Digital Academic Repository) Learning to Transform, Combine, and Reason in Open-domain Question Answering (unknown): “Most of the previous approaches focused on removing noise in the retrieval steps of the question answering pipeline [4, 14, 15, 19, 34].”
Answer Complex Questions: Path Ranker Is All You Need (2021, background): “The experimental results in two modes have shown that EPR can enhance top-1 (best path for answer extraction) PEM by at least 6% so that it can improve the order of the best path to a higher level.”
Evidence Extraction for Machine Reading Comprehension with Deep Probabilistic Logic (2018, methodology): “We didn’t find any public available sentence selector that run-able, inspired by Htut et al. (2018), 9https://github.com/commonsense/conceptnet5 10https://github.com/cjhutto/vaderSentiment 11https://github.com/shuohangwang/comatch we use InferSent (Conneau et al., 2017), which has show competitive…”
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, methodology): “[94] explore two different passage rankers that assign scores to retrieved passages based on their likelihood of containing the answer to a given question.”