A Cache-Based Natural Language Model for Speech Recognition

By Roland Kuhn, R. Mori. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Unbounded cache model for online language modeling with open vocabulary (2017, background): “A popular approach adds a cache to the model and has shown early success in the context of speech recognition [36, 38, 37].”
Improving Neural Language Models with a Continuous Cache (2016, methodology): “A cache model stores a simple representation of the recent past, often in the form of unigrams, and uses them for prediction (Kuhn & De Mori, 1990).”
MAESTRO: Matched Speech Text Representations through Modality Matching (2022, neutral): “One approach uses multi-task training which trains a single model with different objectives for each modality [5, 10, 11], while another uses modality conversion where text is directly converted to speech [12] via text-to-speech (TTS) approaches or viceversa [13] prior to self-supervised training.”
Efficient softmax approximation for GPUs (2016, background): “In particular, smoothed N-gram models [2, 23, 24] achieve good performance in practice [30], especially when they are associated with cache models [26].”
A bit of progress in language modeling (2001, background): “Caching models (Kuhn, 1988; Kuhn & De Mori, 1990; Kuhn &De Mori, 1992) make use of the observation that if you use a word, you are likely to use it again.”
Direct Output Connection for a High-Rank Language Model (2018, neutral): “Grave et al. (2017) extended a cache model (Kuhn and De Mori, 1990) for RNN language models.”
Modeling Local Coherence: An Entity-Based Approach (2005, background): “Cache language models (Kuhn and Mori, 1990) seem particularly promising in this context.”
Query Expansion with Locally-Trained Word Embeddings (2016, methodology): “al model is adapted) online using the topic-specic training data. A topic-dependent model may be combined with the global model using linear interpolation [21] or other more sophisticated approaches [14, 24]. Similarly to the adaptation work, we use topic-specic documents to train a topicspecic mode”
Multiscale recurrent neural network based language model (2015, unknown): “1 for the cache-based LM [11], where such weights were determined on the basis of the perplexity of the validation data.”
Modeling long distance dependence in language: topic mixtures vs. dynamic cache models (1996, methodology): “One approach to overcome this limitation is to use dynamic cache language models [l, 2, 31, which model task-level dependencies by inmasing the likelihood of a word given that it has been observed previously.”
NIRMAL: Automatic identification of software relevant tweets leveraging language model (2015, unknown): “[10], [12], [13], [30].”