Multi-style Generative Reading Comprehension

By Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, Junji Tomita. 2019

71 citations recorded by the source corpus.

This study tackles generative reading comprehension (RC), which consists of answering questions based on textual evidence and natural language generation (NLG). We propose a multi-style abstractive summarization model for question answering, called Masque. The proposed model has two key characteristics. First, unlike most studies on RC that have focused on extracting an answer span from the provided passages, our model instead focuses on generating a summary from the question and multiple passages. This serves to cover various answer styles required for real-world applications. Second, whereas previous studies built a specific model for each answer style because of the difficulty of acquiring one general model, our approach learns multi-style answers within a model to improve the NLG capability for all styles involved. This also enables our model to give an answer in the target style. Experiments show that our model achieves state-of-the-art performance on the Q&A task and the Q&A + NLG task of MS MARCO 2.1 and the summary task of NarrativeQA. We observe that the transfer of the style-independent NLG capability to the target style is the key to its success.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In Natural Language Processing

Citations in the corpus

PALM: Pre-training an Autoencoding&autoregressive Language Model for Context-conditioned Generation (2020, methodology): “PALM incorporates the copy mechanism by plugging in the pointer-generator network (See et al., 2017b; Nishida et al., 2019) decoder in Transformer.”
Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021, background): “Nishida et al. (2019a) proposed a multi-style generative model to generate an abstractive summary from the given question, passages and multi-style.”
A Transformer-based Approach for Source Code Summarization (2020, methodology): “We use an additional attention layer to learn the copy distribution on top of the decoder stack (Nishida et al., 2019).”
Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering (2020, methodology): “On the NarrativeQA test set, we improve Rouge-L by 3.0 over (Nishida et al., 2019).”