RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation

By Christoph Lüscher, Eugen Beck, Kazuki Irie, Markus Kitza, Wilfried Michel, Albert Zeyer, Ralf Schlüter, Hermann Ney. 2019

240 citations recorded by the source corpus.

We present state-of-the-art automatic speech recognition (ASR) systems employing a standard hybrid DNN/HMM architecture compared to an attention-based encoder-decoder design for the LibriSpeech task. Detailed descriptions of the system development, including model design, pretraining schemes, training schedules, and optimization approaches are provided for both system architectures. Both hybrid DNN/HMM and attention-based systems employ bi-directional LSTMs for acoustic modeling/encoding. For language modeling, we employ both LSTM and Transformer based architectures. All our systems are built using RWTHs open-source toolkits RASR and RETURNN. To the best knowledge of the authors, the results obtained when training on the full LibriSpeech training set, are the best published currently, both for the hybrid DNN/HMM and the attention-based systems. Our single hybrid system even outperforms previous results obtained from combining eight single systems. Our comparison shows that on the LibriSpeech 960h task, the hybrid DNN/HMM system outperforms the attention-based system by 15% relative on the clean and 40% relative on the other test sets in terms of word error rate. Moreover, experiments on a reduced 100h-subset of the LibriSpeech training corpus even show a more pronounced margin between the hybrid DNN/HMM and attention-based architectures.

Subfields: Artificial Intelligence, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

Citations in the corpus

Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, neutral): “d test-other Word Error Rate (WER). Method clean other Iterations seq2seq Zeyer et al. (2018a) 4.9 15.4 n Zeyer et al. (2018b) 4.7 15.2 n Irie et al. (2019) 4.7 13.4 n Sabour et al. (2019) 4.5 13.3 n Luscher et al. (2019) 4.4 13.5 n Park et al. (2019) 4.1 12.5 n ASG/CTC Collobert et al. (2016) 7.2 -”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, methodology): “Our best models are transformer-based, as [5, 6], which gave good results in Seq2Seq even without external language models [18].”
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020, neutral): “9 100h labeled Hybrid DNN/HMM [34] 4-gram 5.”
You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation (2020, neutral): “We also put results for train-clean-100 from RETURNN [1] as the strongest hybrid baseline known to us.”
Effectiveness of self-supervised pre-training for speech recognition (2019, methodology): “(Irie et al., 2019; Kahn et al., 2019; Lscher et al., 2019) evaluated semi-supervised self-labeling WER performance on the standard test “clean” and test “other” benchmarks of the Librispeech dataset (Panayotov et al., 2015) when using only 100 hour subset as labeled data.”
Going deeper with Image Transformers (2021, background): “They became a staple in speech recognition too [45, 36], being it in encoder and sequence criterion or encoder-decoder seq2seq [61] conformations, and hold the state of the art to this day [74, 78] with models 36 blocks deep.”
Deep Contextualized Acoustic Representations for Semi-Supervised Speech Recognition (2019, methodology): “Note that while [27] introduced SpecAugment to significantly improve LibriSpeech performance via data augmentation, and [28] achieved state-of-the-art results using both hybrid and end-toend models, our approach focuses on the SSL case with less labeled training data via our DeCoAR features.”
A Comparative Study on Transformer vs RNN in Speech Applications (2019, neutral): “Our transformer results are comparable to the best performance in [26], [44], [45].”
Transformer-Based Acoustic Modeling for Hybrid Speech Recognition (2019, background): “3 We also average the last 10 epoch checkpoints to form an extra candidate. ing [38], the best checkpoints for test-clean and test-other are selected separately on the corresponding development sets 4”
End-to-End Speech Recognition: A Survey (2023, methodology): “10, on Librispeech, the current best-published classical hybrid systems range around 2.3% (test-clean) and 4.9% (test-other) word error rate [222], [323], while there already are a number of E2E systems providing similar performance [205], [206], [224], [320], with some E2E systems clearly…”
Improving Speech Recognition Using Consistent Predictions on Synthesized Speech (2020, background): “[22] and [24] report competitive performance with 4.”
Non-Autoregressive Transformer Automatic Speech Recognition (2019, unknown): “In recent studies very deep end-to-end automatic speech recognition (ASR) starts to match and outperform conventional ASR systems [1, 2, 3].”