S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension

By Chuanqi Tan, Furu Wei, Nan Yang, Weifeng Lv, M. Zhou. 2017

92 citations recorded by the source corpus.

In this paper, we present a novel approach to machine reading comprehension for the MS-MARCO dataset. Unlike the SQuAD dataset that aims to answer a question with exact text spans in a passage, the MS-MARCO dataset defines the task as answering a question from multiple passages and the words in the answer are not necessary in the passages. We therefore develop an extraction-then-synthesis framework to synthesize answers from extraction results. Specifically, the answer extraction model is first employed to predict the most important sub-spans from the passage as evidence, and the answer synthesis model takes the evidence as additional features along with the question and passage to further elaborate the final answers. We build the answer extraction model with state-of-the-art neural networks for single passage reading comprehension, and propose an additional task of passage ranking to help answer extraction in multiple passages. The answer synthesis model is based on the sequence-to-sequence neural networks with extracted evidences as features. Experiments show that our extraction-then-synthesis method outperforms state-of-the-art methods.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Simple and Effective Multi-Paragraph Reading Comprehension (2017, background): “The recent success of neural models at answering questions given a related para-graph (Wang et al., 2017b; Tan et al., 2017) suggests neural models have the potential to be a key part of a solution to this problem.”
PALM: Pre-training an Autoencoding&autoregressive Language Model for Context-conditioned Generation (2020, background): “…neural machine translation (Bahdanau et al., 2015; Vaswani et al., 2017), abstractive summarization (Rush et al., 2015; See et al., 2017a; Gehrmann et al., 2018), generative question answering (QA) (Tan et al., 2017; Bi et al., 2019) and conversational response generation (Vinyals and Le, 2015).”
Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering (2017, methodology): “S-Net (Tan et al., 2017) proposes a two-step approach for generative QA.”
The price of debiasing automatic metrics in natural language evalaution (2018, methodology): “We collected annotations on the systems generated by the fastqa and fastqa ext from Weissenborn et al. (2017) and the snet and snet.ens (emble) models from Tan et al. (2018), along with reference answers.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “…coverage (Suzuki and Nagata, 2016; Chen et al., 2016; See et al., 2017), RL based metric optimization (Paulus et al., 2018), graph-based attention (Tan et al., 2017), and the copy mechanism (Miao and Blunsom, 2016; Gu et al., 2016; See et al., 2017). important information been generated); while…”
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension (2018, background): “Note that (Tan et al., 2017) recently proposed an answer generation model for the MS MARCO dataset.”
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (2018, methodology): “We tune the hyper-parameters according to the Model ROUGE-L BLEU-1 FastQA Ext (Weissenborn et al., 2017) 33.67 33.93 Prediction (Wang and Jiang, 2016) 37.33 40.72 ReasoNet (Shen et al., 2017) 38.81 39.86 R-Net (Wang et al., 2017c) 42.89 42.22 S-Net (Tan et al., 2017) 45 Two simple yet effective tech”
Incorporating External Knowledge into Machine Reading for Generative Question Answering (2019, methodology): “S-Net (Tan et al., 2018): An extraction-thensynthesis framework to synthesize answers from extracted evidences”
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering (2020, background): “, 2017), with further improvements using joint learning (Wang et al., 2018a; Tan et al., 2018).”
A Generative Approach to Question Answering (2017, methodology): “We also include the results reported in Table 6 of Tan et al. (2017) on experimenting with basic sequence-to-sequence model with selected passage as the input.”
Human Behavior Inspired Machine Reading Comprehension (2019, unknown): “Although a number of machine reading comprehension (MRC) models [33, 36, 37, 41] have shown the ability to approach or even transcend human beings in a few simplified MRC tasks like SQuAD [26], they are still far behind human beings in a number of challenging MRC tasks, such as MS MARCO [23] and DuR”
HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering (2019, background): “There exist two sorts of approaches, generate answer based on the entail word vocabulary (Tan et al. 2018) and retrieve answer from the current paragraph.”
An Abstractive approach to Question Answering (2017, background): “Tan et al. (2017) has taken a generative approach where they add a decoder on top of their extractive model and thus leveraging the extracted evidence that the extraction module produce to synthesize the answer.”
Multi-style Generative Reading Comprehension (2019, methodology): “Also, non-pipelined models (including ours) consider all the provided passages and find the answer by comparing scores between passages (Tan et al., 2018; Wu et al., 2018).”
On Knowledge distillation from complex networks for response prediction (2019, unknown): “…most state of the art models (Hermann et al., 2015; Kadlec et al., 2016; Cao et al., 2016; Xiong et al., 2016; Seo et al., 2016; Gong and Bowman, 2017; Dhingra et al., 2017; Wang et al., 2017; Shen et al., 2017; Trischler et al., 2016; Group and Asia, 2017; Tan et al., 2017; Sordoni et al., 2016).”
Rethinking Search: Making Experts out of Dilettantes (2021, background): “Historically, this has been thought of as significantly more challenging than extractive or retrieval-based QA [43, 80, 87].”
Focused Hierarchical RNNs for Conditional Sequence Processing (2018, background): “Span-based vs Generative Most of the recent question and answering models for MS MARCO are span-based5 (Weissenborn et al., 2017; Tan et al., 2017; Shen et al., 2017; Wang & Jiang, 2016).”
Entity-Relation Extraction as Multi-Turn Question Answering (2019, methodology): “Tan et al. (2017) train the passage ranking model jointly with the reading comprehension model.”