Dynamic Entity Representation with Max-pooling Improves Machine Reading

By Sosuke Kobayashi, Ran Tian, Naoaki Okazaki, Kentaro Inui. 2016

41 citations recorded by the source corpus.

We propose a novel neural network model for machine reading, DER Network, which explicitly implements a reader building dynamic meaning representations for entities by gathering and accumulating information around the entities as it reads a document. Evaluated on a recent large scale dataset (Hermann et al., 2015), our model exhibits better results than previous research, and we find that max-pooling is suited for modeling the accumulation of information on entities. Further analysis suggests that our model can put together multiple pieces of information encoded in different sentences to answer complicated questions. Our code for the model is available at https://github. com/soskek/der-network

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Question Answering & Reading Comprehension

Citations in the corpus

A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task (2016, neutral): “Concurrently with our paper, Kadlec et al. (2016) and Kobayashi et al. (2016) also experiment on these two datasets and report competitive results.”
Text Understanding with the Attention Sum Reader Network (2016, methodology): “Subﬁgure (b) shows the overall distribution of the number of candidate answers in the news datasets. l l l l l l l l l 0.5 mance of our single model is a little bit worse than performance of simultaneously published models (Chen et al., 2016; Kobayashi et al., 2016).”
ReasoNet: Learning to Stop Reading in Machine Comprehension (2016, neutral): “alid test Deep LSTM Reader [7] 55.0 57.0 63.3 62.2 Attentive Reader [7] 61.6 63.0 70.5 69.0 MemNets [8] 63.4 66.8 - - AS Reader [9] 68.6 69.5 75.0 73.9 Stanford AR [3] 72.2 72.4 76.9 75.8 DER Network [12] 71.3 72.9 - - Iterative Attention Reader [21] 72.6 73.3 - - EpiReader [25] 73.4 74.0 - - GA Rea”
Gated-Attention Readers for Text Comprehension (2016, background): “Other related works include Dynamic Entity Representation network (DER) (Kobayashi et al., 2016), which builds dynamic representations of the candidate answers while reading the document, and accumulates the information about an entity by max-pooling; EpiReader (Trischler et al.”
End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension (2016, methodology): “This observation also agrees with (Kobayashi et al. 2016).”
Embracing data abundance: BookTest Dataset for Reading Comprehension (2016, methodology): “…et al., 2016b; Cui et al., 2016b; Cui et al., 2016a; Li et al., 2016), however some do not explicitly depend on this property (Hermann et al., 2015; Kobayashi et al., 2016; Chen et al., 2016; Weissenborn, 2016) 3 Current Landscape We will now brieﬂy review what datasets for text comprehension have…”
Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations (2018, background): “Kobayashi et al. (2016, 2017) proposed to extract and organize information about each entity in a discourse using the context.”
Learning to Select, Track, and Generate for Data-to-Text (2019, background): “This module is often used in natural language understanding to keep track of the entity state (Kobayashi et al., 2016; Hoang et al., 2018; Bosselut et al., 2018).”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, methodology): “Kobayashi et al. (2016) used a pooling operation to aggregate entity information across multiple mentions.”
From Particular to General: A Preliminary Case Study of Transfer Learning in Reading Comprehension (2016, background): “As a case study we focus on the domain of text comprehension – a domain that would undoubtedly form an important sub-module of a general AI and that has lately attracted a lot of attention in the NLP community [7, 9, 11, 10, 3, 23, 6, 25, 26, 5, 4, 13, 21].”