Lattice rescoring strategies for long short term memory language models in speech recognition

By Shankar Kumar, Michael Nirschl, Daniel Niels Holtmann-Rice, Hank Liao, Ananda Theertha Suresh, Felix X. Yu. 2017

41 citations recorded by the source corpus.

Recurrent neural network (RNN) language models (LMs) and Long Short Term Memory (LSTM) LMs, a variant of RNN LMs, have been shown to outperform traditional N-gram LMs on speech recognition tasks. However, these models are computationally more expensive than N-gram LMs for decoding, and thus, challenging to integrate into speech recognizers. Recent research has proposed the use of lattice-rescoring algorithms using RNNLMs and LSTMLMs as an efficient strategy to integrate these models into a speech recognition system. In this paper, we evaluate existing lattice rescoring algorithms along with new variants on a YouTube speech recognition task. Lattice rescoring using LSTMLMs reduces the word error rate (WER) for this task by 8% relative to the WER obtained using an N-gram LM.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Tara N. Sainath
3. Rohit Prabhavalkar
4. Chung-Cheng Chiu
5. Bo Li
6. Yonghui Wu
7. Ruoming Pang
8. Dong Yu

Citations in the corpus

• AdaCliP: Adaptive Clipping for Private SGD (2019, background): “Most state-of-the-art machine learning models are trained on user data, examples include keyboard models [6], automatic video transcription [7] among others.”
• Two-Pass End-to-End Speech Recognition (2019, methodology): “Language model rescoring is commonly been used for multi-pass decoding [14, 15, 16, 17], but more recently has been used with a LAS model to rescore hypotheses from a first-pass conventional model [18].”
• FedBoost: Communication-Efficient Algorithms for Federated Learning (2020, unknown): “For example, the size of typical state-of-the-art server-side language models is in the order of several hundreds of megabytes (Kumar et al., 2017).”
• LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring (2019, background): “Examples for this approach are [15, 16, 17, 18, 19, 20] where a variety of heuristics is proposed to speed up the rescoring process.”
• Domain Adaptation Using Factorized Hidden Layer for Robust Automatic Speech Recognition (2018, background): “State-of-the-art ASR systems use deep neural networks (DNNs) for acoustic modeling [1, 2] and/or language modeling [3].”