Denoising Distantly Supervised Open-Domain Question Answering

By Yankai Lin, Haozhe Ji, Zhiyuan Liu, Maosong Sun. 2018

151 citations recorded by the source corpus.

Distantly supervised open-domain question answering (DS-QA) aims to find answers in collections of unlabeled text. Existing DS-QA models usually retrieve related paragraphs from a large-scale corpus and apply reading comprehension technique to extract answers from the most relevant paragraph. They ignore the rich information contained in other paragraphs. Moreover, distant supervision data inevitably accompanies with the wrong labeling problem, and these noisy data will substantially degrade the performance of DS-QA. To address these issues, we propose a novel DS-QA model which employs a paragraph selector to filter out those noisy paragraphs and a paragraph reader to extract the correct answer from those denoised paragraphs. Experimental results on real-world datasets show that our model can capture useful information from noisy data and achieve significant improvements on DS-QA as compared to all baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Improving Question Answering with External Knowledge (2019, methodology): “trieving relevant documents in Wikipedia and then running a MRC model to extract a text span from the documents based on the question (Chen et al., 2017; Wang et al., 2018b; Kratzwald and Feuerriegel, 2018; Lee et al., 2018; Lin et al., 2018).”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “Speciﬁcally, improvements include stronger reading comprehension models (Wang et al., 2018; Yang et al., 2019; Xiong et al., 2019; Min et al., 2019a) and paragraph reranking models (Wang et al., 2017; Lin et al., 2018) that assign more accurate relevance scores to the retrieved paragraphs.”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “The subsequent work improves the heuristic retriever by re-ranking retrieved documents (Wang et al., 2018a;b; Lee et al., 2018; Lin et al., 2018).”
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering (2019, background): “Effect of passage granularity : Previous work usually deﬁnes passages as articles (Chen et al., 2017), paragraphs (Yang et al., 2019), or sentences (Wang et al., 2018a; Lin et al., 2018).”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “tant supervision (Lin et al., 2018), or multi-task learning (Nishida et al.”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, methodology): “Open-domain QA systems such as R 3 (Wang et al., 2018a) and DS-QA (Lin et al., 2018) have sophisticated retriever models where the reader and retriever are jointly trained.”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, background): “Specific recent applications in question answering include Bordes et al. (2015), Chen et al. (2017), Lin et al. (2018), as well as Joshi et al. (2017) for building benchmark test collections.”
R2-D2: A Modular Baseline for Open-Domain Question Answering (2021, neutral): “Lin et al. (2018) proposed to denoise distantly supervised answer string matches in MML via paragraph-ranker.”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “…et al., 2019), but uses two times more text at inference time.13 Two con- 12For multiple blocks, 20-best answer spans are generated from each block and answer string probabilities are weighted by the retriever block probability and aggregated (summed) across blocks, following Lin et al. (2018).”
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, methodology): “…for CosmosQA, for each answer, the input token sequence is modiﬁed as “ < SEP > context BiDAF (Seo et al., 2016) 28.60 34.60 25.90 28.50 - AQA (Buck et al., 2018) 40.50 47.40 - - - Rˆ3 (Wang et al., 2017a) 49.00 55.30 35.30 41.70 - DSQA (Lin et al., 2018) 49.00 55.30 42.30 49.30 - Evidence Agg.”
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension (2019, background): “…state-of-the-art performance on ﬁvedatasets, although it does not even 1 State-of-the-are-results were found in (Tay et al., 2018) for N EWS QA, in Lin et al. (2018), for S EARCH QA, in Das et al. (2019) for TQA-U, in (Talmor and Berant, 2018b) for CWQ, in Ding et al. (2019) for H OTPOT QA, in…”
UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021, background): “In (Lin et al., 2018), a coarse-to-ﬁne model is proposed to handle label noise by aggregating information from relevant paragraphs and then extracting answers from selected ones.”
Open-Domain Question Answering (2020, background): “, 2018; Nogueira and Cho, 2019), and denoising distantly-supervised data (Lin et al., 2018).”
Simple and Efficient ways to Improve REALM (2021, background): “However, such a signal can lead to noisy and unrelated documents to be given a positive signal (Lin et al., 2018).”