Estimation of probabilities from sparse data for the language model component of a speech recognizer

By Slava M. Katz. 1987

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Speech & Audio

Citations in the corpus

LLaMA: Open and Efficient Foundation Language Models (2023, background): “, 1983), and various smoothing techniques were proposed to improve the estimation of rare events (Katz, 1987; Kneser and Ney, 1995).”
A bit of progress in language modeling (2001, methodology): “While the most commonly used smoothing techniques, Katz smoothing (Katz, 1987) and Jelinek-Mercer smoothing (Jelinek and Mercer, 1980) (sometimes called deleted interpolation) work fine, even better smoothing techniques exist.”
The Design for the Wall Street Journal-based CSR Corpus (1992, neutral): “Lincoln has also provided 8 baseline bigram back-off[8,11] language models (5K/20K words × open/closed vocab.”
Exponentially Increasing the Capacity-to-Computation Ratio for Conditional Computation in Deep Learning (2014, methodology): “Beside the obvious regularization of choosing small k , we propose here an additional device that is inspired by the impressive success of smoothed or interpolated n-grams and back-off models in statistical language modeling (Katz, 1987; Jelinek and Mercer, 1980).”
Gmail Smart Compose: Real-Time Assisted Writing (2019, methodology): “We therefore choose an n-gram language model with Katz-backoff [29] for this purpose.”
Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces (2018, methodology): “The generalization properties of the ASR engine are preserved by using a statistical n-gram LM [21] allowing to mix parts of the training queries to create new ones, and by using class-based language modeling [9] where the value of each entity may be replaced by any other.”
Entropy-based Pruning of Backoff Language Models (2000, background): “N-gram backoff models [5], despite their shortcomings, still dominate as the technology of choice for state-of-the art speech recognizers [4].”
Extending Neural Generative Conversational Model using External Knowledge Sources (2018, methodology): “In natural language processing, RNNs are used to learn language models that generalize over n-gram models (Katz, 1987).”
Headline Generation Based on Statistical Translation (2000, methodology): “Prob-abilities for sequences that have not been seen in the training data are estimated using back-off weights (Katz, 1987).”
Composition in Distributional Models of Semantics (2010, methodology): “As indicated, Experiment 1 is based on a sentence similarity task initially proposed by Kintsch (2001). In his study, Kintsch investigates how the meaning of an ambiguous verb, for example run, changes in the context of specific noun subjects, for example horse or colour.”
A Fixed-Size Encoding Method for Variable-Length Sequences with its Application to Neural Network Language Models (2015, methodology): “Traditionally, the back-off n-gram models [Katz, 1987, Kneser, 1995] are the standard approach to language modeling.”