Dynamic Coattention Networks For Question Answering

By Caiming Xiong, Victor Zhong, Richard Socher. 2016

698 citations recorded by the source corpus.

Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (2018, background): “In spite of a variety of model structures and attention types (Cui et al., 2016; Xiong et al., 2016; Seo et al., 2016; Wang et al., 2017; Clark and Gardner, 2017), a typical attention-based neural network model for MRC ﬁrst encodes the symbolic representation of the question and passage in an…”
U-Net: Machine Reading Comprehension with Unanswerable Questions (2018, background): “…Currently, end-to-end neural network models have achieved great successes for machine reading comprehension (Seo et al. 2016; Kumar et al. 2015; Sukhbaatar et al. 2015; Cui et al. 2016; Xiong, Zhong, and Socher 2016; Dhingra et al. 2016; Shen et al. 2016; Hu et al. 2017; Wang, Yan, and Wu 2018).”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, methodology): “2017), question answering (Seo et al., 2016; Xiong et al., 2016), and natural language inference1.”
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, methodology): “To measure the speedup of our model against the RNN models, we also test the corresponding model architecture with each encoder block replaced with a stack of bidirectional Published 12 LeaderBoard 13 Single Model EM / F1 EM / F1 LR Baseline (Rajpurkar et al., 2016) 40.4 / 51.0 40.4 / 51.0 Dynamic C”
Learned in Translation: Contextualized Word Vectors (2017, methodology): “In order to test the transferability of these encoders, we develop a common architecture for a variety of classification tasks, and we modify the Dynamic Coattention Network for question answering [Xiong et al., 2017].”
NumNet: Machine Reading Comprehension with Numerical Reasoning (2019, neutral): “…Reader (Kadlec et al., 2016), BiDAF (Seo et al., 2017), Interactive AoA Reader (Cui et al., 2017), Gated Attention Reader (Dhingra et al., 2017), R-Net (Wang et al., 2017a), DCN (Xiong et al., 2017), QANet (Yu et al., 2018), and achieve promising results in most existing public MRC datasets.”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Noting this, we extend the coattention of (Xiong et al., 2017) to enrich the representation of not only the input but also the question.”
Unifying Question Answering and Text Classification via Span Extraction (2019, methodology): “For question answering, a task-speciﬁc span-decoder is often used to extract a span of text verbatim from a portion of the input text (Xiong et al., 2016).”
Option Comparison Network for Multiple-choice Reading Comprehension (2019, background): “n a more ﬂexible way. 2.3 Article Rereading Mimicking humans, the article will be reread with the option correlation features as extra input to gain deeper understanding. Specially, the co-attention (Xiong et al., 2017) and selfattention (Wang et al., 2017) mechanisms are adopted for rereading. Firs”
Reading Wikipedia to Answer Open-Domain Questions (2017, background): “Test EM F1 EM F1 Dynamic Coattention Networks (Xiong et al., 2016) ﬁnd that while these help for more exact paragraph reading in SQuAD, they don’t improve results in the full setting.”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “, 2015) is a synthetic dataset that captures various aspects of reasoning; and SQuAD (Rajpurkar et al., 2016; Wang et al., 2016; Xiong et al., 2016) and NewsQA (Trischler et al.”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “…multi-task learning of (1) reading comprehension, that extracts an answer span from a reasoning path E using a standard approach (Seo et al., 2017; Xiong et al., 2017; Devlin et al., 2019), and (2) reasoning path re-ranking, that re-ranks the retrieved reasoning paths by computing the probability…”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, background): “A lot of effort has been put into designing sophisticated neural MRC architectures for reading short context (e.g. a single paragraph), with much success (Wang & Jiang, 2017; Seo et al., 2017; Xiong et al., 2017; Wang et al., 2018c; Yu et al., 2018, inter alia).”
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (2018, background): “With the introduction of large scale machine comprehension datasets, machine comprehension models that are highly accurate and efﬁcient in answering questions given raw texts have been proposed recently (Seo et al., 2016; Xiong et al., 2016; Wang et al., 2017c).”
Zero-Shot Relation Extraction via Reading Comprehension (2017, background): “The task of obtaining answer spans to natural-language questions has been recently studied on the SQuAD dataset (Rajpurkar et al., 2016; Xiong et al., 2016; Lee et al., 2016; Wang et al., 2016).”
Towards VQA Models That Can Read (2019, methodology): “This approach has been used for a variety of tasks in NLP such as summarization [11, 34, 39], question answering [46], language modelling [32], neural machine translation [12], and dialog [37].”
Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, methodology): “Recently, various neural network models were proposed and successfully applied to the tasks of questions answering (QA) and/or reading comprehension (Xiong et al., 2016; Dhingra et al., 2016; Yang et al., 2017).”
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning (2017, methodology): “Pointer models have also been successfully applied to tasks such as language modeling (Merity et al., 2017), summarization (Gu et al., 2016), combinatorial optimization (Bello et al., 2017), and question answering (Seo et al., 2017; Xiong et al., 2017).”
Gated Self-Matching Networks for Reading Comprehension and Question Answering (2017, methodology): “Xiong et al. (2016) and Seo et al. (2016) employ variant coattention mechanism to match the question and passage mutually.”