Simple and Effective Semi-Supervised Question Answering

By Bhuwan Dhingra, Danish Pruthi, Dheeraj Rajagopal. 2018

67 citations recorded by the source corpus.

Recent success of deep learning models for the task of extractive Question Answering (QA) is hinged on the availability of large annotated corpora. However, large domain specific annotated corpora are limited and expensive to construct. In this work, we envision a system where the end user specifies a set of base documents and only a few labelled examples. Our system exploits the document structure to create cloze-style questions from these base documents; pre-trains a powerful neural network on the cloze style questions; and further fine-tunes the model on the labeled examples. We evaluate our proposed system across three diverse datasets from different domains, and find it to be highly effective with very little labeled data. We attain more than 50% F1 score on SQuAD and TriviaQA with less than a thousand labelled examples. We are also releasing a set of 3.2M cloze-style questions for practitioners to use while building QA systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “Finally, we compare our semi-supervised QA methods with Dhingra et al. (2018).”
Unsupervised Commonsense Question Answering with Self-Talk (2020, background): “Motivations vary from data augmentation to QA tasks (Du et al., 2017; Dhingra et al., 2018; Du and Cardie, 2018; Sachan and Xing, 2018) through conversational machine reading (Saeidi et al.”
Unsupervised Question Answering by Cloze Translation (2019, methodology): “We ﬁnd our approach performs best in very data poor regimes, and similarly to Dhingra et al. (2018) with modest amounts of data.”
Training Question Answering Models from Synthetic Data (2020, methodology): “Earlier work (Dhingra et al., 2018; Lewis et al., 2019) using named entity and noun phrase answer candidates performed best only on those portions of the data distribution.”
Multi-Stage Pretraining for Low-Resource Domain Adaptation (2020, background): “Golub et al. (2017), Yang et al. (2017), Lewis et al. (2019) and Dhingra et al. (2018) develop approaches to generate natural questions that can aid downstream question answering tasks.”
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text (2018, background): “As a result, models which operate on these patterns (e.g. BiDAF (Seo et al., 2017)) do not generalize beyond their training domains (Wiese et al., 2017; Dhingra et al., 2018) or to novel types of reasoning (Welbl et al., 2018; Talmor and Berant, 2018).”
A Survey on Deep Semi-Supervised Learning (2021, unknown): “Semantic role labeling CoNLL-2009 [127], CoNLL-2013 [128] [129], [130], [131] Question answering SQuAD [132], TriviaQA [133] [134], [135], [136]”
Teaching Machine Comprehension with Compositional Explanations (2020, background): “(1) “Distant” Supervision: to generate “proxy” training examples automatically (Dhingra et al., 2018; Lewis et al., 2019; Li et al., 2020); (2) Learning Efficiency: a model learns quickly with minimal supervision (Radford et al.”
Span Selection Pre-training for Question Answering (2019, methodology): “We also compare our span selection pre-training data with the data distributed by Dhingra et al. (2018). This data consists of approximately 2 million instances constructed using the abstract and body structure of Wikipedia.”
Relation-Guided Pre-Training for Open-Domain Question Answering (2021, unknown): “To address the problem of context-question lexical overlap, Dhingra et al. (2018) assume each article has an introductory paragraph, and use this paragraph to generate answer.”
Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision (2021, unknown): “Recent unsupervised MRC studies mainly adopt data synthetization approachs (Yang and Mitchell, 2017; Dhingra et al., 2018; Pan et al., 2020; Hong et al., 2020; Kang et al., 2020).”