An Empirical Study of Smoothing Techniques for Language Modeling

2015

Citations in the corpus

• Librispeech: An ASR corpus based on public domain audio books (2015, methodology): “Modified Kneser-Ney smoothed 3- a nd 4-grams [21, 22] are trained.”
• Character-Aware Neural Language Models (2015, methodology): “A language model is formalized as a probability distribution over a sequence of strings (words), and traditional methods usually involve making an n -th order Markov assumption and estimating n -gram probabilities via counting and subsequent smoothing (Chen and Goodman 1998).”
• Decoding with Large-Scale Neural Language Models Improves Translation (2013, methodology): “The baselines used conventional 5-gram LMs, estimated with modified Kneser-Ney smoothing (Chen and Goodman, 1998) on the English side of the bitext and the 329M-word Xinhua portion of English Gigaword (LDC2011T07).”
• The effect of word predictability on reading time is logarithmic (2013, methodology): “Here we overcome these limitations by combining two large behavioral datasets of word-by-word reading times with probabilistic language models from computational linguistics (Chen & Goodman, 1998; Kneser & Ney, 1995; Manning & Schütze, 1999) and nonparametric statistical analysis.”
• Generating Natural-Language Video Descriptions Using Text-Mined Knowledge (2013, unknown): “These triplets are then used to train a backoff language model with Kneser-Ney smoothing (Chen and Goodman, 1999) for estimating the likelihood of an SVO triple.”
• Maybe Deep Neural Networks are the Best Choice for Modeling Source Code (2019, methodology): “Therefore standard methods from NLP of dealing with this problem, such as introducing a single special token to represent OOV tokens [17], still fall short of fully addressing the OOV problem for code.”