Stochastic Answer Networks for Machine Reading Comprehension

By Xiaodong Liu, Yelong Shen, Kevin Duh, Jianfeng Gao. 2017

202 citations recorded by the source corpus.

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, the unique feature is the use of a kind of stochastic prediction dropout on the answer module (final layer) of the neural network during the training. We show that this simple trick improves robustness and achieves results competitive to the state-of-the-art on the Stanford Question Answering Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading COmprehension Dataset (MS MARCO).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (2018, methodology): “Most of the top-ranking methods on SQuAD leaderboard are based on this attention mechanism (Wang et al., 2017; Huang et al., 2017; Xiong et al., 2017; Liu et al., 2017b,a).”
Deep Contextualized Word Representations (2018, background): “…pretrained word vectors (Turian et al., 2010; Mikolov et al., 2013; Pennington et al., 2014) are a standard component of most state-of-the-art NLP architectures, including for question answering (Liu et al., 2017), textual entailment (Chen et al., 2017) and semantic role labeling (He et al., 2017).”
Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding (2019, methodology): “…to improve model generalization, and has been used to achieve new state-of-the-art results in a wide range of natural language understanding (NLU) tasks, including question answering and machine reading comprehension (Devlin et al., 2018; Liu et al., 2018; Huang et al., 2017; Hancock et al., 2019).”
Multi-Task Deep Neural Networks for Natural Language Understanding (2019, methodology): “Two such examples use the SAN answer module for the pairwise text classiﬁcation output module, and the pairwise ranking loss for the QNLI task which by design is a binary classiﬁcation 5 There is an ongoing discussion on revising the QNLI dataset.”
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension (2018, methodology): “To overcome this limitation, we filtered out triples whose queries can be easily answered by the stateof-the-art MRC architecture, Stochastic Answer Networks (SAN) (Liu et al., 2018).”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “…hold for question answering in general, so recent models for SQuAD are not necessarily general question answering models [Yu et al., 2018a, Hu et al., 2018, Wang et al., 2017a, Liu et al., 2017b, Huang et al., 2017, Xiong et al., 2018, Liu et al., 2017a, Pan et al., 2017, Salant and Berant, 2017].”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, background): “Models that do iterative reasoning (Shen et al., 2017; Liu et al., 2017) i.”
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (2018, methodology): “petitive architectures (Liu et al., 2018; Clark and Gardner, 2017; Wang et al., 2017; Seo et al., 2017; Pan et al., 2017; Salant and Berant, 2018; Xiong et al., 2018) on SQuAD can be viewed as similar to our implemented model.”
Neural Approaches to Conversational AI (2018, positive): “ years, attracting signiﬁcant attention from the QA research community. 29 Figure 3.8: Two examples of state of the art neural MRC models. (Left) The Stochastic Answer Net (SAN) model. Figure credit: Liu et al. (2018d). (Right) The BiDirectional Attention Flow (BiDAF) model. Figure credit: Seo et al”
Errudite: Scalable, Reproducible, and Testable Error Analysis (2019, background): “…for particular slices of interest (e.g., accuracy over question types in machine comprehension, per-label performance in semantic role labeling) (Liu et al., 2017; He et al., 2017), and (2) counterfactual error analysis , where one modiﬁes the input data to assess if expectations are met, such…”
Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation (2018, neutral): “The tasks include sentiment analysis (Socher et al. 2013), entailment (Bowman et al. 2015), summarization (Nallapati, Xiang, and Zhou 2016), question answering (Liu et al. 2018), and semantic role labeling (He et al. 2017).”
Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog (2019, methodology): “Inspired by [31], in the third strategy, during training, we randomly dropout the results of some steps with a ﬁxed probability, and use the average of the remaining steps for prediction.”
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading (2019, methodology): “Machine Reading Comprehension: MRC models such as SQuAD-like models, aim to extract answer spans (starting and ending indices) from a given document for a given question (Seo et al., 2017; Liu et al., 2018b; Yu et al., 2018).”
Contextualized Word Representations for Reading Comprehension (2017, methodology): “aSoR + TR [7] 75.8 83.3 DCN+ [8] 75.1 83.1 Conductor-net [9] 73.2 81.9 RaSoR (base model) [10] 70.8 78.7 Table 2: Single-model results on SQuAD’s test set.3 [1]Peters et al.(2018) [2,7] This work. [3]Liu et al. (2017b) [4]Wang et al.(2017) [5]Huang et al.(2017) [6]Cui et al.(2017) [8]Xiong et al.(20”