Sequence-to-Sequence Asr Optimization Via Reinforcement Learning

By Andros Tjandra, Sakriani Watiasri Sakti, Satoshi Nakamura. 2017

27 citations recorded by the source corpus.

Despite the success of sequence-to-sequence approaches in automatic speech recognition (ASR) systems, the models still suffer from several problems, mainly due to the mismatch between the training and inference conditions. In the sequence-to-sequence architecture, the model is trained to predict the grapheme of the current time-step given the input of speech signal and the ground-truth grapheme history of the previous time-steps. However, it remains unclear how well the model approximates real-world speech during inference. Thus, generating the whole transcription from scratch based on previous predictions is complicated and errors can propagate over time. Furthermore, the model is optimized to maximize the likelihood of training data instead of error rate evaluation metrics that actually quantify recognition quality. This paper presents an alternative strategy for training sequence-to-sequence ASR models by adopting the idea of reinforcement learning (RL). Unlike the standard training scheme with maximum likelihood estimation, our proposed approach utilizes the policy gradient algorithm. We can (1) sample the whole transcription based on the model's prediction in the training process and (2) directly optimize the model with negative Levenshtein distance as the reward. Experimental results demonstrate that we significantly improved the performance compared to a model trained onlv with maximum likelihood estimation.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Optimal Completion Distillation for Sequence Learning (2018, methodology): “In terms of CER, our work substantially outperforms prior work as compared in Table 4, with the closest being Tjandra et al. (2018) trained with policy gradients on CER.”
End-to-End Speech Recognition: A Survey (2023, background): “Also, there have been studies that consider MWER in terms of reinforcement learning [140], [141].”