Broad Context Language Modeling as Reading Comprehension

By Kevin Gimpel, Hai Wang, David A. McAllester, Zewei Chu. 2016

25 citations recorded by the source corpus.

Progress in text understanding has been driven by large datasets that test particular capabilities, like recent datasets for reading comprehension (Hermann et al., 2015). We focus here on the LAMBADA dataset (Paperno et al., 2016), a word prediction task requiring broader context than the immediate sentence. We view LAMBADA as a reading comprehension problem and apply comprehension models based on neural networks. Though these models are constrained to choose a word from the context, they improve the state of the art on LAMBADA from 7.3% to 49%. We analyze 100 instances, finding that neural network readers perform well in cases that involve selecting a name from the context based on dialogue or discourse cues but struggle when coreference resolution or external knowledge is needed.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Universal Transformers (2018, methodology): “In the latter setting, introduced by Chu et al. Chu et al. (2017), the target sentence (minus the last word) is used as query for selecting the target word from the context sentences.”
A Simple Method for Commonsense Reasoning (2018, methodology): “higher accuracy. Unsupervised training is done on text corpora which can be cheaply curated. Using language models in reading comprehension tests also produced many great successes. Namely Chu et al. [28] used bi-directional RNNs to predict the last word of a passage in the LAMBADA challenge. Simila”
Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension (2019, background): “, 2017), word prediction (Chu et al., 2017), and judgment prediction (Long et al.”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, methodology): “Analysis of the questions (Chu et al., 2017) suggests that around 20% of the questions need coreference understanding to answer correctly.”
CliCR: a Dataset of Clinical Case Reports for Machine Reading Comprehension (2018, result): “Our ﬁndings align with those of Chu et al. (2017) on the Lambada dataset (Paperno et al., 2016): Although they used a different categorization of comprehension skills, they also ﬁnd that GA reader has most difﬁculties with elaboration (which they refer to as “external knowledge”), followed by…”
Linguistic Knowledge as Memory for Recurrent Neural Networks (2017, unknown): “Chu et al. (2016) also performed manual analysis on a sub-set of the test set and found that approximately 20% of the passages required coreference resolution to ﬁnd the correct answer.”
Neural Shufﬂe-Exchange Networks − Sequence Processing in O( n log n ) Time (2019, methodology): “In 81% cases of the test set the target word can be found in the text and we follow a common strategy (Chu et al., 2017; Dehghani et al., 2018) to choose the target word as one from the text.”