QURIOUS: Question Generation Pretraining for Text Generation
By Shashi Narayan, Gonçalo Simães, Ji Ma, Hannah Craighead, Ryan T. McDonald. 2020
16 citations recorded by the source corpus.
Recent trends in natural language processing using pretraining have shifted focus towards pretraining and fine-tuning approaches for text generation. Often the focus has been on task-agnostic approaches that generalize the language modeling objective. We propose question generation as a pretraining method, which better aligns with the text generation objectives. Our text generation models pretrained with this method are better at understanding the essence of the input and are better language models for the target task. When evaluated on two text generation tasks, abstractive summarization and answer-focused question generation, our models result in state-of-the-art performances in terms of automatic metrics. Human evaluators also found our summaries and generated questions to be more natural, concise and informative.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bernd Bohnet
- Joshua Maynez
- Nan Ding
- Soravit Changpinyo
- Xi Chen
- Ryan T. McDonald
- Mohit Bansal
- Shashi Narayan
In Computer Vision and Pattern Recognition
- Soravit Changpinyo
- Radu Soricut
- Xi Chen
- Nan Ding
- Idan Szpektor
- Luc Van Gool
- Shashi Narayan
- Ryan T. McDonald
In Natural Language Processing
- Bernd Bohnet
- Ryan T. McDonald
- Joshua Maynez
- Shashi Narayan
- Idan Szpektor
- Radu Soricut
- Soravit Changpinyo
- Nan Ding
Citations in the corpus
- On Faithfulness and Factuality in Abstractive Summarization (2020, background): “Better ways of generating questions (Narayan et al., 2020) and measuring fac-”
- Quiz-Style Question Generation for News Stories (2021, unknown): “leveraging weak supervision via an intermediate training stage to further refine pre-trained models have also been explored in previous works in question generation and headline generation [14, 32].”
- All You May Need for VQA are Image Captions (2022, background): “There are two main directions to QG: template-based (Heil-man and Smith, 2009; Lyu et al., 2021; Dhole and Manning, 2020) and neural-based, with the latter achieving state-of-the-art results (Alberti et al., 2019; Narayan et al., 2020).”