Improving Attention-Based End-to-End ASR Systems with Sequence-Based Loss Functions

By Jia Cui, Chao Weng, Guangsen Wang, J. Wang, Peidong Wang, Chengzhu Yu, Dan Su, Dong Yu. 2018

10 citations recorded by the source corpus.

Acoustic model and language model (LM) have been two major components in conventional speech recognition systems. They are normally trained independently, but recently there has been a trend to optimize both components simultaneously in a unified end-to-end (E2E) framework. However, the performance gap between the E2E systems and the traditional hybrid systems suggests that some knowledge has not yet been fully utilized in the new framework. An observation is that the current attention-based E2E systems could produce better recognition results when decoded with LMs which are independently trained with the same resource.In this paper, we focus on how to improve attention-based E2E systems without increasing model complexity or resorting to extra data. A novel training strategy is proposed for multi-task training with the connectionist temporal classification (CTC) loss. The sequence-based minimum Bayes risk (MBR) loss is also investigated. Our experiments on SWB 300hrs showed that both loss functions could significantly improve the baseline model performance. The additional gain from joint-LM decoding remains the same for CTC trained model but is only marginal for MBR trained model. This implies that while CTC loss function is able to capture more acoustic knowledge, MBR loss function exploits more word/character dependency.

Subfields: Artificial Intelligence, Optimization & Training, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Chao Weng
6. Ron J. Weiss
7. Yoshua Bengio
8. Shinji Watanabe

In Optimization & Training

1. Jinyu Li
2. Dong Yu
3. Chao Weng
4. Tara N. Sainath
5. Peidong Wang
6. Jia Cui
7. Ron J. Weiss
8. Vaibhava Goel

In Speech & Audio

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Chao Weng
5. Ron J. Weiss
6. Peidong Wang
7. Jia Cui
8. Shinji Watanabe

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Chao Weng
6. Ron J. Weiss
7. Yifan Gong
8. Shinji Watanabe

Citations in the corpus

• Multitask Training with Text Data for End-to-End Speech Recognition (2020, background): “Various model architectures [6, 7, 8, 9, 10] and training schemes [11, 12, 13] were proposed.”
• Token-wise Training for Attention Based End-to-end Speech Recognition (2019, methodology): “comparing various sequence discriminative training criteria for A-E2E systems [11].”
• Large Margin Training for Attention Based End-to-End Speech Recognition (2019, unknown): “A common approximation of posterior graph is the n-best hypotheses list [9, 10, 12].”
• Recent Advances in End-to-End Automatic Speech Recognition (2021, methodology): “MWER has been shown to improve the accuracy of AED models [124, 125], RNN-T models [126, 127], and hybrid autoregressive transducer (HAT) models [128].”