End-to-end Speech Recognition Using Lattice-free MMI

By Hossein Hadian, Hossein Sameti, Daniel Povey, Sanjeev Khudanpur. 2018

171 citations recorded by the source corpus.

We present our work on end-to-end training of acoustic models using the lattice-free maximum mutual information (LF-MMI) objective function in the context of hidden Markov models. By end-to-end training, we mean ﬂat-start training of a single DNN in one stage without using any previously trained models, forced alignments, or building state-tying decision trees. We use full biphones to enable context-dependent modeling without trees, and show that our end-to-end LF-MMI approach can achieve comparable results to regular LF-MMI on well-known large vocabulary tasks. We also compare with other end-to-end methods such as CTC in character-based and lexicon-free settings and show 5 to 25 percent relative reduction in word error rates on different large vocabulary tasks while using signiﬁ-cantly smaller models.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “Nonetheless, there have been increasingly more end-to-end, competitive results, trained with CTC [7, 8], ASG [9], LF-MMI [10], sequence-to-sequence (Seq2Seq) [11, 12], or transducers training [13, 14], and differentiable decoding [15].”
Rethinking Evaluation in ASR: Are Our Models Robust Enough? (2020, methodology): “All the Transformer LMs share the similar architecture as [32]’s Google Billion Words model: we use 8 attention heads; 8 (WSJ, CV and SB+FSH), 16 (TL) or 20 (LS) decoder layers with embedding, input and output dimensions of 512 (CV), 1024 (WSJ and 2https://github.com/flashlight/flashlight SB+FHS) or”
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, neutral): “, 0.0125) and (0.1, 0.025). Table 5: Switchboard 300h WERs (%). Method No LM With LM SWBD CH SWBD CH HMM Vesel´y et al., (2013) [41] 12.9 24.5 Povey et al., (2016) [30] 9.6 19.3 Hadian et al., (2018) [42] 9.3 18.9 Zeyer et al., (2018) [24] 8.3 17.3 CTC Zweig et al., (2017) [43] 24.7 37.1 14.0 25.3 A”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, neutral): “5 Lattice-free MMI (Hadian et al., 2018) - 5.”
wav2vec: Unsupervised Pre-training for Speech Recognition (2019, neutral): “Our best pre-training model performs as well as the phoneme-based model of Hadian et al. (2018).”
Fully Convolutional Speech Recognition (2018, neutral): “pproach but a mel-ﬁlterbanks front-end and a 4-gram language model. 1https://github.com/facebookresearch/wav2letter 2https://github.com/facebookresearch/fairseq Model dev93 nov92 E2E Lattice-free MMI [27] - 4.1 (data augmentation) CNN-DNN-BLSTM-HMM [19] 6.6 3.5 (speaker adaptation, 3k acoustic state”
Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, background): “al’s definition of end-to-end [14]: “flatstart training of a single DNN in one stage without using any previously trained models, forced alignments, or building state-tying decision trees.”
Speech Recognition (2018, unknown): “In comparison to Hadian et al. (2018), wav2vec performs as well as their phoneme-based model and wav2vec large outperforms it by 0.37 WER.”
From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition (2019, background): “The decoding parameters are tuned to minimize WER on the development set.”
Who Needs Words? Lexicon-Free Speech Recognition (2019, background): “To the best of our knowledge, the first ASR system to achieve competitive results with a character-based LM and without a lexicon (on Switchboard and WSJ) was [20], that our lexiconfree character-based ConvLM surpasses on WSJ (see Table 4).”
A Fully Differentiable Beam Search Decoder (2019, methodology): “hough, with (ﬁxed) token level (phone) language models and no lexicon, as found in lattice-free MMI (LF-MMI) (Povey et al.,2016), and even trained end-to-end without any alignments (EE-LF-MMI) as in (Hadian et al., 2018), with a pre-trained phone LM still. ASG maximizes Equation (9) which is similar”