Advancing Sequence-to-Sequence Based Speech Recognition

By Zolt'an Tuske, Kartik Audhkhasi, George Saon. 2019

25 citations recorded by the source corpus.

The paper presents our endeavor to improve state-of-the-art speech recognition results using attention based neural network approaches. Our test focus was LibriSpeech, a well-known, publicly available, large, speech corpus, but the methodologies are clearly applicable to other tasks. After systematic application of standard techniques – sophisticated data augmentation, various dropout schemes, scheduled sampling, warm-restart –, and optimizing search configurations, our model achieves 4.0% and 11.7% word error rate (WER) on the test-clean and testother sets, without any external language model. A powerful recurrent language model drops the error rate further to 2.7% and 8.2%. Thus, we not only report the lowest sequence-tosequence model based numbers on this task to date, but our single system even challenges the best result known in the literature, namely a hybrid model together with recurrent language model rescoring. A simple ROVER combination of several of our attention based systems achieved 2.5% and 7.3% WER on the clean and other test sets.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, neutral): “…since different authors use different data augmentation methods (e.g., speed perturbation (Li et al., 2019), tempo/volume perturbation (Tuske et al., 2019), SpecAug-ment (Park et al., 2019 We built strong CTC baselines with the architecture described above, which outperform prior…”
End-to-End Speech Recognition: A Survey (2023, methodology): “This includes the use of full label context dependency both for E2E systems [180], [44], [187], [250], [145], [219], [251], [252], as well as classical systems via full-context language models [253], [197], [198], [254].”