Probabilistic Models of Short and Long Distance Word Dependencies in Running Text

By Julian Kupiec. 1989

This article describes two complementary models that represent dependencies between words in local and non-local contexts. The type of local dependencies considered are sequences of part of speech categories for words. The non-local context of word dependency considered here is that of word recurrence, which is typical in a text. Both are models of phenomena that are to a reasonable extent domain independent, and thus are useful for doing prediction in systems using large vocabularies.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Unbounded cache model for online language modeling with open vocabulary (2017, background): “A popular approach adds a cache to the model and has shown early success in the context of speech recognition [36, 38, 37].”
Improving Neural Language Models with a Continuous Cache (2016, methodology): “Adding a cache to a language model was intoducted in the context of speech recognition(Kuhn, 1988; Kupiec, 1989; Kuhn & De Mori, 1990).”
A bit of progress in language modeling (2001, background): “This observation is the basis of caching (Kuhn, 1988; Kuhn and De Mori, 1990; Kuhn and De Mori, 1992; Kupiec, 1989; Jelinek et al., 1991).”
Adaptive Language Modeling Using Minimum Discriminant Estimation (1992, methodology): “The idea of using a window of the previous N words, called a cache, to estimate dynamic frequencies for a word was proposed in [5] for the case of a tri-part-ofspeech model and in [ 6 ] for a bigram model.”
Two decades of statistical language modeling: where do we go from here? (2000, background): “Cache LMs were first introduced by [59] and [60].”