Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks

By Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, Qifa Ke. 2018

278 citations recorded by the source corpus.

Question generation, the task of automatically creating questions that can be answered by a certain span of text within a given passage, is important for question-answering and conversational systems in digital assistants such as Alexa, Cortana, Google Assistant and Siri. Recent sequence to sequence neural models have outperformed previous rule-based systems. Existing models mainly focused on using one or two sentences as the input. Long text has posed challenges for sequence to sequence neural models in question generation – worse performances were reported if using the whole paragraph (with multiple sentences) as the input. In reality, however, it often requires the whole paragraph as context in order to generate high quality questions. In this paper, we propose a maxout pointer mechanism with gated self-attention encoder to address the challenges of processing long text inputs for question generation. With sentence-level inputs, our model outperforms previous approaches with either sentence-level or paragraph-level inputs. Furthermore, our model can effectively utilize paragraphs as inputs, pushing the state-of-the-art result from 13.9 to 16.3 (BLEU_4).

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Unified Language Model Pre-training for Natural Language Understanding and Generation (2019, methodology): “We identify the approximate extractive oracle for training using the method described in (Zhou et al., 2018).”
PALM: Pre-training an Autoencoding&autoregressive Language Model for Context-conditioned Generation (2020, methodology): “, 2019), question generation (Zhao et al., 2018) and conversational response generation (Vinyals and Le, 2015).”
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “Baselines First, as shown in Table 1, our base-line QG model obtains a non-trivial improvement over previous best QG system (Zhao et al., 2018) which proves the effectiveness of our newly introduced setups: introduce POS/NER features, use deep contexturalized word vectors (from ELMo or BERT), and…”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “A long line of prior work has trained generative models for question answering (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Yang et al., 2020; Bartolo et al., 2021; Lewis et al., 2021).”
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training (2020, methodology): “, 2019) is based on pretrained models, while the other three methods are sequenceto-sequence models enhanced with manual features (Du & Cardie, 2018), gated self-attention (Zhao et al., 2018), and reinforcement learning (Zhang & Bansal, 2019).”
Generating Fact Checking Briefs (2020, neutral): “Unlike previous question generation models (Du et al., 2017; Duan et al., 2017; Tang et al., 2017; Zhao et al., 2018) that generate based on an answer, we treat question generation closer to structured planning — laying out the format for the entire brief.”
Unsupervised Question Answering by Cloze Translation (2019, background): “Recently there has been interest in question generation using supervised neural models, many trained to generate questions from c, a pairs in SQuAD (Du et al., 2017; Yuan et al., 2017; Zhao et al., 2018; Du and Cardie, 2018; Hosking and Riedel, 2019)”
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021, background): “Many approaches have been proposed to generate question-answer pairs given a passage (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Lewis et al., 2021b).”
G-DAug: Generative Data Augmentation for Commonsense Reasoning (2020, methodology): “A candidate solution that has shown promise in other tasks, such as reading comprehension, is to augment a human-authored training set with a large set of synthetically-generated examples (Zhou et al., 2017; Du et al., 2017; Zhao et al., 2018a).”
Transformer-based End-to-End Question Generation (2020, methodology): “The most prevalent technique for question generation studies is the usage of a sequence-to-sequence (Seq2Seq) model (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Dong et al., 2019) in addition to a variety of other features and mechanisms.”
Low-Resource Generation of Multi-hop Reasoning Questions (2020, unknown): “…answer-focused questions; (c) CorefNQG (Du and Cardie, 2018) that incorporated linguistic features to represent the inputs better; (d) MaxPointer (Zhao et al., 2018) using gated self-attention to form questions for long text inputs; (e) MPQG+R (Song et al., 2018) that captured a broader context…”
Quiz-Style Question Generation for News Stories (2021, unknown): “There have also been various extensions, including copy or pointer mechanisms to overcome out-of-vocabulary issues [11, 44], mechanisms for handling paragraph-length inputs [50, 51], and more [19, 33].”
Putting the Horse before the Cart: A Generator-Evaluator Framework for Question Generation from Text (2018, background): “In the context of QG from paragraphs, (Zhao et al., 2018) proposed maxout pointer network to keep track of word coverage .”
A Recurrent BERT-based Model for Question Generation (2019, methodology): “For handling long context, the work (Zhao et al., 2018) introduces a maxout pointer mechanism with a gated self-attention encoder for processing paragraph-level input.”
Review-based Question Generation with Adaptive Instance Transfer and Augmentation (2019, background): “…or service have become an important information source and there are a few research areas analyzing such data, including aspect extraction (Bing et al., 2016; Chen et al., 2013), product recommendation (Chelliah and Sarkar, 2017), and sentiment analysis (Li et al., 2018; Zhao et al., 2018a).”
Learning to Generate Questions by Learning to Recover Answer-containing Sentences (2021, methodology): “Moreover, ASGen consistently improves the performance when applied to other question generation models such as Zhao et al. (2018), T5 (Small and Large), and UniLM across all metrics for both splits. In particular, applying ASGen on UniLM further improves its question generation capability, achieving”