Natural Language Comprehension with the EpiReader

By Adam Trischler, Zheng Ye, Xingdi Yuan, Philip Bachman, Alessandro Sordoni, Kaheer Suleman. 2016

95 citations recorded by the source corpus.

We present the EpiReader, a novel model for machine comprehension of text. Machine comprehension of unstructured, real-world text is a major research goal for natural language processing. Current tests of machine comprehension pose questions whose answers can be inferred from some supporting text, and evaluate a model's response to the questions. The EpiReader is an end-to-end neural model comprising two components: the first component proposes a small set of candidate answers after comparing a question to its supporting text, and the second component formulates hypotheses using the proposed candidates and the question, then reranks the hypotheses based on their estimated concordance with the supporting text. We present experiments demonstrating that the EpiReader sets a new state-of-the-art on the CNN and Children's Book Test machine comprehension benchmarks, outperforming previous neural models by a significant margin.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

NewsQA: A Machine Comprehension Dataset (2016, methodology): “Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al., 2016a).”
Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering (2017, methodology): “On cloze-test tasks (Hermann et al., 2015), Epireader Trischler et al. (2016) relates to our work in the sense that it is a two-step extractor-reasoner model, which ﬁrst extracts K most probable single-token answer candidates and then constructs a hypothesis by combining each answer candidate to the”
Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, background): “Although larger question answering datasets with hundreds of thousands of question-answer pairs have been collected, including SQuAD (Rajpurkar et al., 2016), MSMARCO (Nguyen et al., 2016), and NewsQA (Trischler et al., 2016a), the data collection process is expensive and time-consuming in practice.”
Gated Self-Matching Networks for Reading Comprehension and Question Answering (2017, methodology): “…with cloze-style datasets, several powerful deep learning models (Hermann et al., 2015; Hill et al., 2016; Chen et al., 2016; Kadlec et al., 2016; Sordoni et al., 2016; Cui et al., 2016; Trischler et al., 2016; Dhingra et al., 2016; Shen et al., 2016) have been introduced to solve this problem.”
Machine Comprehension Using Match-LSTM and Answer Pointer (2016, methodology): “Given that answers often come from the given passage, Pointer Network has been adopted in a few studies in order to copy tokens from the given passage as answers (Kadlec et al., 2016; Trischler et al., 2016).”
ReasoNet: Learning to Stop Reading in Machine Comprehension (2016, methodology): “[25] propose the EpiReader which uses two neural network structures: one extracts candidates using the attention-sum reader; the other reranks candidates based on a bilinear term similarity score calculated from query and passage representations.”
Gated-Attention Readers for Text Comprehension (2016, background): “, 2016), which builds dynamic representations of the candidate answers while reading the document, and accumulates the information about an entity by max-pooling; EpiReader (Trischler et al., 2016) consists of two networks, where one proposes a small set of candidate answers, and the other reranks t”
Attention-over-Attention Neural Networks for Reading Comprehension (2016, methodology): “We also added pretty new works to compare with (Dhingra et al., 2016; Sordoni et al., 2016; Trischler et al., 2016), which came to our attention when we are writing this paper.”
Tracking the World State with Recurrent Entity Networks (2016, neutral): “However, (Trischler et al., 2016) 0.697 0.674 AoA Reader (Cui et al., 2016) 0.720 0.694 NSE Adaptive Computation (Munkhdalai & Yu, 2016) 0.732 0.714 there is still a performance gap when compared to more sophisticated machine comprehension models, many of which perform multiple layers of attention…”
Words or Characters? Fine-grained Gating for Reading Comprehension (2016, background): “A variety of models have been proposed to extract answers from given text (Hill et al., 2016; Kadlec et al., 2016; Trischler et al., 2016; Chen et al., 2016; Sordoni et al., 2016; Cui et al., 2016).”
Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge (2018, neutral): “0 EpiReader (Trischler et al., 2016) 74.”
Subword-augmented Embedding for Cloze Reading Comprehension (2018, background): “Recently, many deep learning models have been proposed for reading comprehension (Sordoni et al., 2016; Trischler et al., 2016; Wang and Jiang, 2016; Munkhdalai and Yu, 2017; Wang et al., 2017a; Dhingra et al., 2017; Zhang et al., 2018b; Wang et al., 2018b).”
Machine Comprehension by Text-to-Text Neural Question Generation (2017, background): “labeled datasets has spurred development of increasingly advanced models for question answering (QA) from text (Kadlec et al., 2016; Trischler et al., 2016b; Seo et al., 2016; Wang et al., 2016; Shen et al., 2016).”
On Knowledge distillation from complex networks for response prediction (2019, unknown): “…most state of the art models (Hermann et al., 2015; Kadlec et al., 2016; Cao et al., 2016; Xiong et al., 2016; Seo et al., 2016; Gong and Bowman, 2017; Dhingra et al., 2017; Wang et al., 2017; Shen et al., 2017; Trischler et al., 2016; Group and Asia, 2017; Tan et al., 2017; Sordoni et al., 2016).”
Human Behavior Inspired Machine Reading Comprehension (2019, methodology): “Inspired by the reread behavior of humans, multi-turn reasoning mechanism has been introduced into MRC models to improve the performance [8, 30, 31, 35, 38].”