Trigger-based language models: a maximum entropy approach

By Raymond Lau, R. Rosenfeld, Salim Roukos. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Unbounded cache model for online language modeling with open vocabulary (2017, background): “Finally, Lau et al. [42] propose to use pairs of distant of words to capture long-range dependencies.”
Improving Neural Language Models with a Continuous Cache (2016, background): “Finally, Lau et al. (1993) proposes to use pairs of distant of words to capture long-range dependencies.”
From captions to visual concepts and back (2014, background): “We have also experimented with distant bigram features [22] and continuous space log-bilinear features [29, 30], but while these improved PPLX significantly, they did not improve BLEU, METEOR or human preference, and space restrictions preclude further discussion.”
Exponential Reservoir Sampling for Streaming Language Models (2014, background): “However, language shows observable priming effects, sometimes called triggers , where the occurrence of a given term decreases the sur-prisal of some other term later in the same discourse (Lau et al., 1993; Church and Gale, 1995; Beeferman et al., 1997; Church, 2000).”
Modeling long distance dependence in language: topic mixtures vs. dynamic cache models (1996, background): “Trigger language models [ 6 ] capture dependencies within a sentence but are computationally very expensive while training the models.”
Statistical language model adaptation: review and perspectives (2004, background): “Still, self-triggers have been shown to be particularly powerful and robust (Lau et al., 1993), which underscores the desirability of exploiting correlations between the current word and features of the large-span history.”
Two decades of statistical language modeling: where do we go from here? (2000, methodology): “For example, in [8], conventional ngrams, distance-2 ngrams, and long distance word pairs (“triggers”) were encoded as features, and resulted in up to 39% perplexity reduction and up to 14% speech recognition word error rate reduction over the trigram baseline.”
Exploiting latent semantic information in statistical language modeling (2000, background): “In this approach, the pair (stocks, fell) is said to form a word trigger pair [51].”
Context dependent recurrent neural network language model (2012, background): “Finally, in whole-sentence language models [20, 21], trigger features based on the existence of widely separated word pairs also provides long-span information.”
Modeling long distance dependence in language: topic mixtures versus dynamic cache models (1999, background): “Trigger language models [ 8 ] as well as decision-tree language models [9] have explored the issue of dependencies within a sentence but are computationally very expensive while training the models.”
A maximum entropy approach to adaptive statistical language modelling (1996, background): “(cid:25) This has been tried, with moderate success only [Lau et al. 93b].”
Statistical Language Models Based on Neural Networks (2012, background): “As the cache models provide truly significant improvements in perplexity (sometimes even more than 20%), there exists a large number of more refined techniques that can capture the same patterns as the basic cache models - for example, various topic models, latent semantic analysis based models [3],”
Whole-sentence exponential language models: a vehicle for linguistic-statistical integration (2001, unknown): “It is instructive to compare the whole-sentence ME model with conditional ME models, which have seen considerable success recently in language modeling [10, 11, 8, 3].”
Large Vocabulary Continuous Speech Recognition : a ReviewSteve (1996, methodology): “As a consequence various alternatives have been studied such as tree-based models[11], trellis models[98], trigger models[63], history models[17] and variable N-grams [25].”