Statistical language model adaptation: review and perspectives

By Jerome R. Bellegarda. Speech Communication, 2004

353 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/98aa7cfc9b1efd016fb6da4e684f027d9fef6f15

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

On the naturalness of software (2016, background): “Even when only a small amount of relevant code exists, language model adaptation, 5 and cacheing 49 could be applied.”
Bayesian Language Model Interpolation for Mobile Speech Input (2011, background): “There is a large literature on language model adaptation, in general, and mixture language models, in particular [6, 7].”
Large language models empowered agent-based modeling and simulation: a survey and perspectives (2023, neutral): “…(Taori et al., 2023), and GLM (Zeng et al., 2023), are the latest paradigm of language models, which evolve from early statistical language models (Bellegarda, 2004) to neural language models (Melis et al., 2017), then to pre-trained language models (Brown et al., 2020), and ﬁ nally to large…”
Continual Adaptation for Efficient Machine Communication (2019, background): “the data distribution are often found across test contexts (Kneser and Steinbiss, 1993; Riccardi and Gorin, 2000; Bellegarda, 2004; Ben-David et al., 2010).”
Multi-domain Neural Network Language Generation for Spoken Dialogue Systems (2016, methodology): “In contrast, model-based adaptation is particularly useful for language modeling (LM) (Bellegarda, 2004).”
Low Rank Language Models for Small Training Sets (2011, methodology): “For domain adaptation, there are many other approaches that could be leveraged [13], and the LRLM might be useful as the filtering LM used in selecting data from out-of-domain sources [14].”
Prosodic and temporal features for language modeling for dialog (2012, background): “If we can accomplish this, the resulting language model could turn out to be more robust than n-grams, which are known to be brittle (Bellegarda, 2004); to the extent that we can represent patterns of word occurrence that reflect fundamental cognitive processes and constraints, such language models…”
On the naturalness of software (2012, methodology): “Even when only a small amount of relevant code exists, language model adaptation techniques [32] could be applied, using corpora of similar code.”
Shrinking Exponential Language Models (2009, methodology): “For a survey of techniques using word classes, topic, syntax, etc., refer to (Bellegarda, 2004).”
Tailored gamification: A review of literature (2020, background): “…to the task (e.g., the difference in writing technical papers and casual e-mails) and the inherent variation of the speech based on diverse aspects (e.g., socio-economic status, emotional states, cultural context) (Bellegarda, 2004) to present the most suitable game elements (Forsyth et al., 2013).”
Dynamic Evaluation of Neural Sequence Models (2017, methodology): “ur again. 4 BACKGROUND 4.1 RELATED APPROACHES Adaptive language modelling was ﬁrst considered for n-grams, adapting to recent history via caching (Jelinek et al., 1991; Kuhn, 1988), and other methods Bellegarda (2004). More recently, the neural cache approach (Grave et al., 2017) and the closely rel”
Recurrent neural network language model adaptation with curriculum learning (2015, background): “The topic was taken up again in the machine learning community by Bengio et al. (2009) under the label ‘curriculum earning’. Here, the interest was not generally on restrictions during the learning process, but rather on restrictions n the input data. Specifically, Bengio et al. (2009) shows that fi”
Recurrent neural network based language model personalization by social network crowdsourcing (2013, background): “Traditionally, LM adaptation [10, 11, 12] stressed the problem of cross-domain linguistic mismatch, in which the cross-individual linguistic mismatch is often ignored.”
The Microsoft 2017 Conversational Speech Recognition System (2017, background): “There is a large body of work on adaptation of language models to topics and otherwise [46], and recurrent neural network model extensions have been proposed to condition on global context [17].”