Advances in all-neural speech recognition

By Geoffry Zweig, Chengzhu Yu, Jasha Droppo, Andreas Stolcke. 2016

96 citations recorded by the source corpus.

This paper advances the design of CTC-based all-neural (or end-to-end) speech recognizers. We propose a novel symbol inventory, and a novel iterated-CTC method in which a second system is used to transform a noisy initial output into a cleaner version. We present a number of stabilization and initialization methods we have found useful in training these networks. We evaluate our system on the commonly used NIST 2000 conversational telephony test set, and significantly exceed the previously published performance of similar systems, both with and without the use of an external language model and decoding technology.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Shinji Watanabe
6. Ralf Schlüter
7. Yu Zhang
8. Rohit Prabhavalkar

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Shinji Watanabe
6. Yu Zhang
7. Ralf Schlüter
8. Rohit Prabhavalkar

Citations in the corpus

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, neutral): “, (2017) [43] 24.”
• Improved training of end-to-end attention models for speech recognition (2018, methodology): “Connectionist temporal classification (CTC) [10] has been often used as an end-to-end model for speech recognition, often on characters/graphemes [11–16] or on sub-words [17] but also directly on words [18,19].”
• Latent Sequence Decompositions (2016, methodology): “Collobert et al. (2016) and Zweig et al. (2016) developed extensions of CTC where they used some word pieces.”
• Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation (2016, neutral): “This type of architecture has been proposed for machine translation (Sutskever et al., 2014; Bahdanau et al., 2015) and for automatic speech recognition (Chorowski et al., 2015; Chan et al., 2016; Zweig et al., 2016).”
• Improving Attention-Based End-to-End ASR Systems with Sequence-Based Loss Functions (2018, background): “1 Iterated CTC + RNN WLM [50] 14.”
• Improved training for online end-to-end speech recognition systems (2017, methodology): “Following [5], we used a label symbol inventory consisting of the individual characters and their doubleletter units.”
• A Comparison of Sequence-to-Sequence Models for Speech Recognition (2017, methodology): “There has been growing interest in building end-to-end trained systems that directly map the input acoustic speech signal to grapheme or word sequences [3, 4, 5, 6, 7, 8, 9, 10, 11, 12].”
• Building Competitive Direct Acoustics-to-Word Models for English Conversational Speech Recognition (2017, background): “The second character set is the one used in [5], and includes separate character variants depending on position in a word - beginning, middle, and end.”
• Recent progresses in deep learning based acoustic models (2017, background): “Other output units that are larger than characters but smaller than words have also been studied [68].”
• An Investigation of Phone-Based Subword Units for End-to-End Speech Recognition (2020, unknown): “As a concrete example, for the Switchboard corpus, it takes more modeling effort for the char CTC system to match performance of the phone CTC system, as noted by [16] and [17].”
• Hierarchical Multitask Learning for CTC-based Speech Recognition (2018, methodology): “We concatenate every two consecutive input vectors (as in [25,26]), which reduces the time resolution by a factor of two and speeds up computation.”
• Direct Acoustics-to-Word Models for English Conversational Speech Recognition (2017, background): “In particular, the CTC loss enables training AMs by mapping acoustics to phone or character sequences without any frame-level alignment [11–17].”
• Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling (2017, methodology): “Following the previous works (Zweig et al., 2016b; Povey et al., 2016; Xiong et al., 2016b; Sercu and Goel, 2016), evaluation is carried out on the NIST 2000 CTS test set, which comprises both Switch-board (SWB) and CallHome (CH) subsets.”