Learning to Ask: Neural Question Generation for Reading Comprehension

By Xinya Du, Junru Shao, Claire Cardie. 2017

741 citations recorded by the source corpus.

We study automatic question generation for sentences from text passages in reading comprehension. We introduce an attention-based sequence learning model for the task and investigate the effect of encoding sentence- vs. paragraph-level information. In contrast to all previous work, our model does not rely on hand-crafted rules or a sophisticated NLP pipeline; it is instead trainable end-to-end via sequence-to-sequence learning. Automatic evaluation results show that our system significantly outperforms the state-of-the-art rule-based system. In human evaluations, questions generated by our system are also rated as being more natural (i.e.,, grammaticality, fluency) and as more difficult to answer (in terms of syntactic and lexical divergence from the original text and reasoning needed to answer).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, positive): “Visual Question Generation (Shukla et al., 2019; Li et al., 2018) 28.”
ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, methodology): “Following previous works [9, 10], we used the SQuAD dataset [33].”
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “, 2014) for QG tasks, taking answer sentence as input and outputting the question (Du et al., 2017; Zhou et al., 2017), which proved to be better than rulebased methods.”
Towards a Better Metric for Evaluating Question Generation Systems (2018, background): “…problem and create even more training data, there is growing interest in developing techniques that can automatically generate questions from a given source, say a document (Du et al., 2017; Du and Cardie, 2017), knowledge base (Reddy et al., 2017; Serban et al., 2016), or image (Li et al., 2017) .”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “A long line of prior work has trained generative models for question answering (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Yang et al., 2020; Bartolo et al., 2021; Lewis et al., 2021).”
The Turking Test: Can Language Models Understand Instructions? (2020, background): “While there is prior work on learning to ask reading comprehension questions from thousands of examples (Du et al., 2017), our goal here is to test how well a model performs when given the same information as human annotators.”
Just Ask: Learning to Answer Questions from Millions of Narrated Videos (2020, positive): “Moreover, rule-based generation requires the manual creation of rules by experts which is expensive, and has also been recently outperformed by neural question generation [20, 89, 99] as used in our approach.”
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021, background): “, 2019), generative modelling for contextual QA (Lewis and Fan, 2018), as well as being studied in its own right (Du et al., 2017; Hosking and Riedel, 2019).”
Unsupervised Commonsense Question Answering with Self-Talk (2020, background): “Motivations vary from data augmentation to QA tasks (Du et al., 2017; Dhingra et al., 2018; Du and Cardie, 2018; Sachan and Xing, 2018) through conversational machine reading (Saeidi et al.”
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (2020, methodology): “Question Generation To instantiate p ( Q | Y ) , we draw on recent work on automatic question generation (QG), which models this distribution using neural seq2seq models (Du et al., 2017; Kr-ishna and Iyyer, 2019).”
Synthetic QA Corpora Generation with Roundtrip Consistency (2019, background): “Question generation is a well-studied task in its own right (Heilman and Smith, 2010; Du et al., 2017; Du and Cardie, 2018).”
Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks (2018, methodology): “(Du et al., 2017) pioneered the work of automatic QG using an end-to-end trainable sequence to sequence neural model.”
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (2018, methodology): “Compared to models that only take into account sentence-level information (Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017), we find that the linguistic knowledge introduced by the coreference representation aids question generation significantly, producing models that outperform the cur”
Identifying Well-formed Natural Language Questions (2018, methodology): “We also demonstrate that such a query well-formedness clas-siﬁer can be used to improve the quality of a sequence-to-sequence question generation model (Du et al., 2017) by showing an improvement of 0.2 BLEU score in its performance ( § 3).”
GLM: General Language Model Pretraining with Autoregressive Blank Infilling (2021, methodology): “Considering the available baseline results, we use the Gigaword dataset (Rush et al., 2015) for abstractive summarization and the SQuAD 1.1 dataset (Rajpurkar et al., 2016) for question generation (Du et al., 2017) as the benchmarks for models pretrained on BookCorpus and Wikipedia.”
Generating Fact Checking Briefs (2020, background): “Unlike previous question generation models (Du et al., 2017; Duan et al., 2017; Tang et al., 2017; Zhao et al., 2018) that generate based on an answer, we treat question generation closer to structured planning — laying out the format for the entire brief.”
Unsupervised Question Answering by Cloze Translation (2019, methodology): “Also, whilst supervised models exist for this task, they require the type of annotation unavailable in this setting (Du et al. 2017; Du and Cardie 2018; Hosking and Riedel 2019, inter alia ).”
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021, background): “Many approaches have been proposed to generate question-answer pairs given a passage (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Lewis et al., 2021b).”
Evaluation of Text Generation: A Survey (2020, methodology): “…the issues found in BLEU and has been widely used for evaluating machine translation models and other models, such as image captioning (Kilickaya et al., 2017), question generation (Nema & Khapra, 2018; Du et al., 2017), and summarization (See et al., 2017; Chen & Bansal, 2018; Yan et al., 2020).”