Semi-Supervised QA with Generative Domain-Adaptive Nets

By Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, William W. Cohen. 2017

158 citations recorded by the source corpus.

We study the problem of semi-supervised question answering—utilizing unlabeled text to boost the performance of question answering models. We propose a novel training framework, the Generative Domain-Adaptive Nets. In this framework, we train a generative model to generate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for training question answering models. We develop novel domain adaptation algorithms, based on reinforcement learning, to alleviate the discrepancy between the model-generated data distribution and the human-generated data distribution. Experiments show that our proposed framework obtains substantial improvement from unlabeled text.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Unsupervised Data Augmentation for Consistency Training (2019, methodology): “Similarly, with the variational auto-encoding framework and reinforce algorithm, classic graphical models based SSL methods with target variable being latent can also take advantage of deep architectures [30, 39, 65].”
RACE: Large-scale ReAding Comprehension Dataset From Examinations (2017, background): “…of effort has been devoted to the machine comprehension task where people aim to construct a system with the ability to ∗* indicates equal contribution answer questions related to a document that it has to comprehend (Chen et al., 2016; Kadlec et al., 2016; Dhingra et al., 2016; Yang et al., 2017).”
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “Some previous works have proposed methods to detect key phrases from a paragraph and automatically extract potential answer spans (Yang et al., 2017; Du and Cardie, 2018; Subramanian et al., 2018).”
Towards a Better Metric for Evaluating Question Generation Systems (2018, background): “Other models focus only on the performance of the QA task (Yang et al., 2017; Duan et al., 2017) and not explicitly on the quality of the generated questions.”
Good Semi-supervised Learning That Requires a Bad GAN (2017, background): “Different from aforementioned approaches, [24] proposes to train conditional generators with adversarial training to obtain complete sample pairs, which can be directly used as additional training cases.”
Synthetic QA Corpora Generation with Roundtrip Consistency (2019, background): “Yang et al. (2017) and Dhingra et al. (2018) both use generated question-answer pairs to improve a QA system, showing large improvements in low-resource settings with few gold labeled examples.”
Improving Machine Reading Comprehension with General Reading Strategies (2018, background): “…answering by randomly reordering words or shufﬂing sentences (Ding and Zhou, 2018; Li and Zhou, 2018) or generating questions through paraphrasing (Yang et al., 2017; Yuan et al., 2017), which require a large amount of training data or limited by the number of training instances (Yu et al., 2018).”
Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption (2020, background): “2017; Vondrick, Pirsiavash, and Torralba 2016), natural language processing (Yang et al. 2017; Dai et al. 2017) and time series synthesis (Donahue, McAuley, and Puckette 2018; Esteban, Hyland, and Rätsch 2017).”
Learning by Asking Questions (2017, methodology): “Such evaluation has also been used in recent works in the language community [54, 56].”
Unsupervised Data Augmentation (2019, methodology): “Similarly, with the variational auto-encoding framework and reinforce algorithm, classic graphical models based SSL methods with target variable being latent can also take advantage of deep architectures [Kingma et al., 2014, Maaløe et al., 2016, Yang et al., 2017].”
TextKD-GAN: Text Generation Using Knowledge Distillation and Generative Adversarial Networks (2019, unknown): “GANs have been exploited recently in various NLP applications such as machine translation [24,25], dialogue models [15], question answering [26], and natural language generation [7,20,19,13,28,29].”
Leveraging Context Information for Natural Question Generation (2018, background): “They either entirely ignore the target answer (Du et al., 2017), or directly hard-code answer positions (Zhou et al., 2017; Yang et al., 2017; Subramanian et al., 2017; Tang et al., 2017; Wang et al., 2017a; Yuan et al., 2017).”
Learning to Collaborate for Question Answering and Asking (2018, background): “On the contrary, Generative Domain-Adaptive Nets (GDAN) (Yang et al., 2017) regards every generated question-answer pair appended with special domain tag as a positive instance.”
An Introduction to Image Synthesis with Generative Adversarial Nets (2018, unknown): “Research of applying GAN in natural language processing is also a growing trend, such as text modeling [13] [22] [23], dialogue generation [24], question answering [25] and neural machine translation [26].”