A Comparison of Sequence-to-Sequence Models for Speech Recognition

By Rohit Prabhavalkar, Kanishka Rao, Tara N. Sainath, Bo Li, Leif M. Johnson, Navdeep Jaitly. 2017

307 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Shinji Watanabe
5. Yu Zhang
6. Yoshua Bengio
7. Li Deng
8. Quoc V. Le

In Natural Language Processing

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Shinji Watanabe
5. Li Deng
6. Yoshua Bengio
7. Ralf Schlüter
8. Hermann Ney

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Shinji Watanabe
5. Yu Zhang
6. Bo Li
7. Li Deng
8. Rohit Prabhavalkar

Citations in the corpus

• Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, methodology): “Three popular types of generative models have been developed for end-to-end speech recognition applications: CTC (Graves et al., 2006; Graves & Jaitly, 2014), RNN-T (Graves, 2012; Graves et al., 2013) and seq2seq (Chan et al., 2016; Bahdanau et al., 2016a).”
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “, 2018a), transduction (Prabhavalkar et al., 2017; He et al., 2019), and differentiable decoding (Collobert et al.”
• Multilingual Speech Recognition with a Single End-to-End Model (2017, methodology): “Our attention-based sequenceto-sequence model is based on the Listen, Attend and Spell (LAS) model [17, 18], the details of which are explained in the next section.”
• A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese (2018, background): “A S] 1 8 M ay 2 01 8 thermore, the modeling units of WPM have been explored in [6], which are sub-word units ranging from graphemes all the way up to entire words.”
• Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese (2018, background): “Recently various modeling units of sequence-to-sequence attention-based models have been studied on English ASR tasks, such as graphemes, CI-phonemes, context-dependent phonemes and word piece models [1, 5, 8].”
• Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition (2017, background): “Our previous work [32] has shown that LAS can approach state-of-the-art performance when trained on larger amounts of data.”
• Exploring architectures, data and units for streaming end-to-end speech recognition with RNN-transducer (2017, methodology): “A defi-ciency of end-to-end systems appears to be in their language modeling capacity [15] which may be because large text-only data are not utilized in end-to-end systems.”
• Improved training of end-to-end attention models for speech recognition (2018, methodology): “Recently, alternative models for end-to-end modeling were also suggested, such as inverted HMMs [39], the recurrent transducer [40–42], or the recurrent neural aligner [43].”
• OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models (2018, methodology): “(ASR) models are built using explicit encoderdecoder topology (Battenberg et al., 2017; Prabhavalkar et al., 2017; Chiu et al., 2017).”
• Model Unit Exploration for Sequence-to-Sequence Speech Recognition (2019, background): “Sequence-to-sequence learning [2] based on encoder-decoder attention models [3] has become popular for both machine translation [4] and speech recognition [5, 6, 7, 8, 9].”
• Monotonic Chunkwise Attention (2017, background): “A detailed comparison of this class of approaches and attention-based models is provided in (Prabhavalkar et al., 2017), where it is shown that attention-based models perform best in speech recognition experiments.”
• Bytes Are All You Need: End-to-end Multilingual Speech Recognition and Synthesis with Bytes (2018, methodology): “The use of graphemes bring model simplicity and enables end-to-end optimization, which has been shown to yield better performance than phoneme-based models [21].”
• Self-supervised Sequence-to-sequence ASR using Unpaired Speech and Text (2019, background): “The model requires a considerable amount of paired speech and text to learn alignment and classification [6, 7], which imposes a constraint when employing the seq2seq in under-resourced domains.”
• WeNet: Production Oriented Streaming and Non-Streaming End-to-End Speech Recognition Toolkit (2021, background): “Recent work [12, 13, 14] also shows that E2E systems have surpassed conventional hybrid ASR systems in the standard of word error rate (WER).”
• Streaming small-footprint keyword spotting using sequence-to-sequence models (2017, methodology): “This model can be thought of as a variant of the RNN-T model augmented with attention, proposed in our previous work [27], wherein we replace the prediction network with an attention-based decoder that computes attention over the targets in the keyword phrase.”
• Recognizing Long-Form Speech Using Streaming End-to-End Models (2019, background): “Examples include connectionist temporal classification (CTC) with character [3, 4] or wordbased [5, 6] targets, the recurrent neural network transducer (RNNT) [7, 8], and attention-based encoder-decoder models [9, 10, 11, 12, 13, 14].”