Towards Online End-to-end Transformer Automatic Speech Recognition
By Emiru Tsunoo, Yosuke Kashiwagi, Toshiyuki Kumakura, Shinji Watanabe. 2019
33 citations recorded by the source corpus.
The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks in end-to-end (E2E) automatic speech recognition (ASR) systems. However, Transformer has a drawback in that the entire input sequence is required to compute self-attention. We have proposed a block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. In this paper, we extend it towards an entire online E2E ASR system by introducing an online decoding process inspired by monotonic chunkwise attention (MoChA) into the Transformer decoder. Our novel MoChA training and inference algorithms exploit the unique properties of Transformer, whose attentions are not always monotonic or peaky, and have multiple heads and residual connections of the decoder layers. Evaluations of the Wall Street Journal (WSJ) and AISHELL-1 show that our proposed online Transformer decoder outperforms conventional chunkwise approaches.
Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Model Architectures
In Speech & Audio
In Signal Processing
Citations in the corpus
- Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss (2020, methodology): “Transformer-based alternatives to RNNs have recently been explored for use in ASR [9, 10, 11, 12].”
- Universal ASR: Unify and Improve Streaming ASR with Full-context Modeling (2020, background): “…et al., 2016), RNN-T models are naturally streamable and have shown great potentials for low-latency streaming ASR (Chang et al., 2019; He et al., 2019; Tsunoo et al., 2019; Sainath et al., 2019; Shen et al., 2019; Li et al., 2020a;b; Sainath et al., 2020; Huang et al., 2020; Moritz et al., 2020).”
- Improving Streaming Automatic Speech Recognition with Non-Streaming Model Distillation on Unsupervised Data (2020, background): “End-to-end streaming models [1, 2, 3, 4, 5, 6, 7] have become attractive for on-device recognition tasks in two aspects: first, endto-end models are usually compact, which makes them suitable to be used on devices.”