Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition

By Chao Weng, Chengzhu Yu, Jia Cui, Chunlei Zhang, Dong Yu. 2019

40 citations recorded by the source corpus.

In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition. Specifically, initialized with a RNN-T trained model, MBR training is conducted via minimizing the expected edit distance between the reference label sequence and on-the-fly generated N-best hypothesis. We also introduce a heuristic to incorporate an external neural network language model (NNLM) in RNN-T beam search decoding and explore MBR training with the external NNLM. Experimental results demonstrate an MBR trained model outperforms a RNN-T trained model substantially and further improvements can be achieved if trained with an external NNLM. Our best MBR trained system achieves absolute character error rate (CER) reductions of 1.2% and 0.5% on read and spontaneous Mandarin speech respectively over a strong convolution and transformer based RNN-T baseline trained on ~21,000 hours of speech.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Shinji Watanabe
3. Dong Yu
4. Tara N. Sainath
5. Takaaki Hori
6. Li Deng
7. Daniel Povey
8. Ralf Schlüter

In Model Architectures

1. Jinyu Li
2. Shinji Watanabe
3. Daniel Povey
4. Sanjeev Khudanpur
5. Tara N. Sainath
6. Jiayu Du
7. Chao Weng
8. Guoguo Chen

In Speech & Audio

1. Jinyu Li
2. Shinji Watanabe
3. Sanjeev Khudanpur
4. Tara N. Sainath
5. Daniel Povey
6. Takaaki Hori
7. Rohit Prabhavalkar
8. Ralf Schlüter

In Signal Processing

1. Jinyu Li
2. Shinji Watanabe
3. Dong Yu
4. Daniel Povey
5. Takaaki Hori
6. Sanjeev Khudanpur
7. Tara N. Sainath
8. Ralf Schlüter

Citations in the corpus

• GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10, 000 Hours of Transcribed Audio (2021, methodology): “This section describes the baseline systems and experimental results for four popular speech recognition toolkits, namely Athena, ESPnet [15], Kaldi [12] and Pika [16].”
• End-to-End Speech Recognition: A Survey (2023, methodology): “Further work includes MWER for RNN-T and self-attention-T [138], as well as MWER using prefix search instead of n-best [139].”
• Recent Advances in End-to-End Automatic Speech Recognition (2021, methodology): “MWER has been shown to improve the accuracy of AED models [124, 125], RNN-T models [126, 127], and hybrid autoregressive transducer (HAT) models [128].”