Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context

By Urvashi Khandelwal, He He, Peng Qi, Dan Jurafsky. 2018

323 citations recorded by the source corpus.

We know very little about how neural language models (LM) use prior linguistic context. In this paper, we investigate the role of context in an LSTM LM, through ablation studies. Specifically, we analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped. On two standard datasets, Penn Treebank and WikiText-2, we find that the model is capable of using about 200 tokens of context on average, but sharply distinguishes nearby context (recent 50 tokens) from the distant history. The model is highly sensitive to the order of words within the most recent sentence, but ignores word order in the long-range context (beyond 50 tokens), suggesting the distant past is modeled only as a rough semantic field or topic. We further find that the neural caching model (Grave et al., 2017b) especially helps the LSTM to copy words from within this distant context. Overall, our analysis not only provides a better understanding of how neural LMs use their context, but also sheds light on recent success from cache-based models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Natural Language Processing

Citations in the corpus

Dissecting Contextual Word Embeddings: Architecture and Representation (2018, background): “Concurrent with our work, Khandelwal et al. (2018) studied the role of context in inﬂuencing language model predictions, Gaddy et al. (2018) analyzed neural constituency parsers, Blevins et al. (2018) explored whether RNNs trained with several different objectives can learn hierarchical syntax, and…”
Character-Level Language Modeling with Deeper Self-Attention (2018, background): “For example Khandelwal et al. [22] ﬁnd that a word-based LSTM language model only effectively uses ˜200 tokens of context (even if more is provided), and that word order only has an effect within the last ˜50 tokens.”
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context (2019, background): “…but not limited to devising novel architectures to better encode the context (Bengio et al., 2003; Mikolov et al., 2010; Zilly et al., 2016; Krause et al., 2016; Grave et al., 2016b; Dauphin et al., 2016; Chung et al., 2016; Merity et al., 2016; Kalchbrenner et al., 2016; Al-Rfou et al.,…”
SCROLLS: Standardized CompaRison Over Long Language Sequences (2022, background): “Some long-context models are evaluated via language modeling perplexity, but this metric mostly captures model sensitivity to local, shortrange patterns (Khandelwal et al., 2018; Sun et al., 1https://www.scrolls-benchmark.com 2021).”
Calibrate Before Use: Improving Few-Shot Performance of Language Models (2021, background): “, 2021), suffer from recency bias (Khandelwal et al., 2018; Ravfogel et al., 2019), and prefer generic responses instead of rare text (Li et al.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “A number of studies evaluated the effect of erasing or masking certain neural network components, such as word embedding dimensions, hidden units, or even full words (Li et al., 2016b; Feng et al., 2018; Khandelwal et al., 2018; Bau et al., 2018).”
What Does BERT Look at? An Analysis of BERT’s Attention (2019, background): “One line of research examines the outputs of language models on carefully chosen input sentences (Linzen et al., 2016; Khandelwal et al., 2018; Gulordava et al., 2018; Marvin and Linzen, 2018).”
Neural Approaches to Conversational AI (2018, neutral): “While the LSTM model has been shown to be effective in encoding textual contexts up to 500 words (Khandelwal et al., 2018), dialogue histories can often be long and there is sometimes a need to exploit longer-term context.”
Linguistic Knowledge and Transferability of Contextual Representations (2019, background): “, 2016a, inter alia), ablating components of the model and dataset (Kuncoro et al., 2017; Gaddy et al., 2018; Khandelwal et al., 2018), or interpreting attention mechanisms (Bahdanau et al.”
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information (2018, methodology): “Popular approaches include visualising the state space of these networks, performing ablations to the network, or using the internal states of the networks for some auxiliary task (e.g., Adi et al., 2016; Kádár et al., 2017; Conneau et al., 2018; Khandelwal et al., 2018).”
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting (2019, methodology): “As an example, [13] shows that language models using LSTM have an effective context size of about 200 tokens on average but are only able to sharply distinguish 50 tokens nearby, indicating that even LSTM struggles to”
Assessing the Benchmarking Capacity of Machine Reading Comprehension Datasets (2019, methodology): “Our shufﬂe-based meth-ods ( s 6 to s 8 ) are inspired by existing analyses for other tasks (Khandelwal et al. 2018; Nie, Wang, and Bansal 2019; Sankar et al. 2019).”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, methodology): “Current NLMs are based on architectures such as recurrent neural networks (RNN) [63], long short-term memory (LSTM) [45], or Transformer [85] that model long range dependencies: a study of LSTM NLMs showed that they use context as large as 250 words [56], much longer than n-grams.”
Dynamic Evaluation of Transformer Language Models (2019, methodology): “LSTMs (Hochreiter and Schmidhuber, 1997) conventionally used for language modeling have been shown to use relatively shorter contexts to make predictions (Khandelwal et al., 2018).”
Guiding Extractive Summarization with Question-Answering Rewards (2019, background): “A recent study by Khandelwal et al. (2018) reported that the recurrent networks are capable of memorizing a recent context of about 20 tokens and the model is highly sensitive to word order, whereas this is less the case for CNN whose max-pooling operation makes it agnostic to word order.”
What Context Features Can Transformer Language Models Use? (2021, background): “Computational issues limit the effective context size such models can be practically trained on, but this size is still sig-niﬁcantly greater the models mentioned above: as previously noted, Khandelwal et al. (2018) revealed inﬂuence from up to 200 tokens of context.”
Controllable Neural Story Plot Generation via Reinforcement Learning (2018, background): “current neural networks (RNNs) using LSTM or GRU cells can theoretically maintain long-term context in their hidden layers, in practice RNNs only use a relatively small part of the history of tokens (Khandelwal et al. 2018). Consequently, stories generated by RNNs tend to lose coherence as the story”
Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting (2020, background): “First, they struggle to preserve very long-term sequential information, which leads to the loss of long-term temporal dependency in time series during the forward path (Khandelwal, He, Qi, & Jurafsky, 2018).”