Learning to Generate Questions by Learning to Recover Answer-containing Sentences

By Seohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee, J. Choo. 2021

19 citations recorded by the source corpus.

To train a question answering model based on machine reading comprehension (MRC), signiﬁcant effort is required to prepare annotated training data composed of questions and their answers from contexts. Recent research has focused on synthetically generating a question from a given context and an annotated (or generated) answer by training an additional generative model to augment the training data. In light of this research direction, we propose a novel pre-training approach that learns to generate contextually rich questions, by recovering answer-containing sentences. We evaluate our method against existing ones in terms of the quality of generated questions, and ﬁne-tuned MRC model accuracy after training on the data synthetically generated by our method. We consistently improve the question generation capability of existing models such as T5 and UniLM, and achieve state-of-the-art results on MS MARCO and NewsQA, and comparable results to the state-of-the-art on SQuAD. Additionally, the data synthetically generated by our approach is beneﬁcial for boosting up the downstream MRC accuracy across a wide range of datasets, such as SQuAD-v1.1, v2.0, KorQuAD and BioASQ, without any modiﬁcation to the existing MRC models. Furthermore, our method shines especially when a limited amount of pre-training or downstream MRC data is given.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

You Only Need One Model for Open-domain Question Answering (2021, methodology): “We generate ‘question-answer’ pairs from a Wikipedia dump us-ing a question and answer generation model trained on the NQ dataset using the ASGen approach (Back et al., 2021).”