Improving End-to-End Speech Recognition with Policy Learning

By Yingbo Zhou, Caiming Xiong, Richard Socher. 2017

40 citations recorded by the source corpus.

Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch between the objective function and metric during training. We show that the above problem can be mitigated by jointly training with maximum likelihood and policy gradient. In particular, with policy learning we are able to directly optimize on the (otherwise non-differentiable) performance metric. We show that joint training improves relative performance by 4% to 13% for our end-to-end model as compared to the same model learned through maximum likelihood. The model achieves 5.53% WER on Wall Street Journal dataset, and 5.42% and 14.70% on Librispeech test-clean and test-other set, respectively.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, neutral): “, (2018) [35] 5.”
Letter-Based Speech Recognition with Gated ConvNets (2017, methodology): “…acoustic models (often ConvNet-based) topped with RNN layers as in (Hannun et al., 2014a; Miao et al., 2015; Saon et al., 2015; Amodei et al., 2016; Zhou et al., 2018; Zeyer et al., 2018) (e.g. a state of the art on WSJ (Chan and Lane, 2015a)), trained with a sequence criterion (the most popular…”
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition (2019, background): “However, these are computationally expensive (Table 1), leading to simplifications like gated recurrent units (GRUs) [8,32]; furthermore, the success of the ReLU(x) = max(0, x) nonlinearity in preventing vanishing gradients enabled the use of vanilla bidirectional recurrent deep neural networks (BRD”
Fully Neural Network Based Speech Recognition on Mobile and Embedded Devices (2018, unknown): “Two-layer 600-dimensional GRU was employed for RNN LM.”
A Fully Differentiable Beam Search Decoder (2019, background): “Also related, several works are targeting the loss/evaluation mismatch (and sometimes exposure bias) through reinforcement learning (policy gradient) (Bahdanau et al., 2016b; Ranzato et al., 2016) even in speech recognition (Zhou et al., 2018).”
Recent Advances in End-to-End Automatic Speech Recognition (2021, background): “2 policy learning, joint training [326] 2018 CTC CNN+bi-GRU 5.”
A Better Variant of Self-Critical Sequence Training (2020, background): “…et al., 2018; Chen et al., 2018; Li and Gong, 2019), reading comprehension(Hu et al., 2017), summarization(Celikyilmaz et al., 2018; Paulus et al., 2017; Wang et al., 2018; Pasunuru and Bansal, 2018), image paragraph generation(Melas-Kyriazi et al., 2018), speech recognition(Zhou et al., 2018).”
Unpaired Image-to-Speech Synthesis With Multimodal Information Bottleneck (2019, methodology): “Speech → Text: We compare our method with four ASR models, DeepSpeech2 [1], Seq2Seq [7], Policy Learning [57], and Gated Convnets [25].”