Gated Self-Matching Networks for Reading Comprehension and Question Answering

By Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, M. Zhou. 2017

673 citations recorded by the source corpus.

In this paper, we present the gated self-matching networks for reading comprehension style question answering, which aims to answer questions from a given passage. We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation. Then we propose a self-matching attention mechanism to refine the representation by matching the passage against itself, which effectively encodes information from the whole passage. We finally employ the pointer networks to locate the positions of answers from the passages. We conduct extensive experiments on the SQuAD dataset. The single model achieves 71.3% on the evaluation metrics of exact match on the hidden test set, while the ensemble model further boosts the results to 75.9%. At the time of submission of the paper, our model holds the first place on the SQuAD leaderboard for both single and ensemble model.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (2018, methodology): “Most of the top-ranking methods on SQuAD leaderboard are based on this attention mechanism (Wang et al., 2017; Huang et al., 2017; Xiong et al., 2017; Liu et al., 2017b,a).”
U-Net: Machine Reading Comprehension with Unanswerable Questions (2018, methodology): “We expected that we could simply use self-attention on the encoded representation H for interaction between question and passage, which contains both bi-attention (Seo et al. 2016) and self-attention (Wang et al. 2017) of the question and passage.”
Deep Contextualized Word Representations (2018, background): “, 2014) are a standard component of most state-ofthe-art NLP architectures, including for question answering (Wang et al., 2017), textual entailment (Chen et al.”
Simple and Effective Multi-Paragraph Reading Comprehension (2017, methodology): “Self-attention has been used in several prior works (Cheng et al., 2016; Wang et al., 2017b; Pan et al., 2017).”
NumNet: Machine Reading Comprehension with Numerical Reasoning (2019, neutral): “…Reader (Kadlec et al., 2016), BiDAF (Seo et al., 2017), Interactive AoA Reader (Cui et al., 2017), Gated Attention Reader (Dhingra et al., 2017), R-Net (Wang et al., 2017a), DCN (Xiong et al., 2017), QANet (Yu et al., 2018), and achieve promising results in most existing public MRC datasets.”
Technical report on Conversational Question Answering (2019, methodology): “turn according to whether each question depends on the conversation history. Many MRC models have been proposed to tackle single-turn QA, such as BiDAF(Seo et al.,2016), DrQA(Chen et al.,2017), R-Net(Wang et al., 2017) and QANet(Yu et al.,2018) in SQuAD dataset. For multi-turn QA, existing models in”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “…hold for question answering in general, so recent models for SQuAD are not necessarily general question answering models [Yu et al., 2018a, Hu et al., 2018, Wang et al., 2017a, Liu et al., 2017b, Huang et al., 2017, Xiong et al., 2018, Liu et al., 2017a, Pan et al., 2017, Salant and Berant, 2017].”
Know What You Don’t Know: Unanswerable Questions for SQuAD (2018, background): “In turn, these datasets have spurred a diverse array of model architecture improvements (Seo et al., 2016; Hu et al., 2017; Wang et al., 2017; Clark and Gard-ner, 2017; Huang et al., 2018).”
Option Comparison Network for Multiple-choice Reading Comprehension (2019, background): “Specially, the co-attention (Xiong et al., 2017) and self-attention (Wang et al., 2017) mechanisms are adopted for rereading.”
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, methodology): “2498 Self-attention A gated self-attention mechanism (Wang et al., 2017) is applied to H to aggregate the long-term dependency within the paragraph.”
Towards Exploiting Background Knowledge for Building Conversation Systems (2018, methodology): “We found this to be a severe limitation of this and other span based models (for example, R-Net (Wang et al., 2017)) .”
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering (2019, background): “Fourth , most effective QA and RC models highly rely on explicit inter-sentence matching between questions and passages (Wang and Jiang, 2017; Wang et al., 2016; Seo et al., 2017; Wang et al., 2017), whereas BERT only applies self-attention layers over the concatenation of a question-passage pair.”
Denoising Distantly Supervised Open-Domain Question Answering (2018, background): “Many reading comprehension systems (Chen et al., 2016; Dhin-gra et al., 2017a; Cui et al., 2017; Shen et al., 2017; Wang et al., 2017) have been proposed and achieved promising results since their multi-layer architectures and attention mechanisms allow them to reason for the question.”
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (2018, methodology): “However, most of the open-domain QA pipelines depend on traditional information retrieval systems which use TF-IDF rankings (Chen et al., 2017; Wang et al., 2017b).”
The NarrativeQA Reading Comprehension Challenge (2017, methodology): “Finally, we see further evidence of the superﬁciality of the questions in the architectures that have evolved to solve them, which tend to exploit span selection based on representations derived from local context and the query (Seo et al., 2016; Wang et al., 2017).”
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (2018, methodology): “…key components is becoming standard practice, and various state-of-the-art or competitive architectures (Liu et al., 2018; Clark and Gardner, 2017; Wang et al., 2017; Seo et al., 2017; Pan et al., 2017; Salant and Berant, 2018; Xiong et al., 2018) on SQuAD can be viewed as similar to our…”
A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning (2019, methodology): “Span To extract the answer either from the passage or from the question, we combine the gating mechanism of Wang et al. (2017) with the standard decoding strategy of Seo et al. (2017) to predict the starting and ending positions across the entire sequence.”