Improving Attention Based Sequence-to-Sequence Models for End-to-End English Conversational Speech Recognition

By Chao Weng, Jia Cui, Guangsen Wang, J. Wang, Chengzhu Yu, Dan Su, Dong Yu. 2018

59 citations recorded by the source corpus.

In this work, we propose two improvements to attention based sequence-to-sequence models for end-to-end speech recognition systems. For the ﬁrst improvement, we propose to use an input-feeding architecture which feeds not only the previous context vector but also the previous decoder hidden state information as inputs to the decoder. The second improvement is based on a better hypothesis generation scheme for sequential minimum Bayes risk (MBR) training of sequence-to-sequence models where we introduce softmax smoothing into N-best generation during MBR training. We conduct the experiments on both Switchboard-300hrs and Switchboard+Fisher-2000hrs datasets and observe signiﬁcant gains from both proposed improvements. Together with other training strategies such as dropout and scheduled sampling, our best model achieved WERs of 8.3%/15.5% on the Switchboard/CallHome subsets of Eval2000 without any external language models which is highly competitive among state-of-the-art English conversational speech recognition systems.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, neutral): “Attention Seq2Seq [36] Y 8.”
Model Unit Exploration for Sequence-to-Sequence Speech Recognition (2019, background): “onemes, sequence-to-sequence 1. INTRODUCTION Sequence-to-sequence learning [1] based on encoder-decoder attention models [2] has become popular for both machine translation [3] and speech recognition [4, 5, 6, 7, 8]. Such models are typically trained to output character-based units: graphemes, byte-”
End-to-End Speech Recognition: A Survey (2023, background): “Another prominent feature of E2E systems besides the alignment property is their direct character-level modeling avoiding phoneme-based modeling and pronunciation lexica [16], [19], [74], [75], [76], [77], [78], [79], [80], [81], [82], with some even heading for whole-word modeling [30], [76].”
Advancing Sequence-to-Sequence Based Speech Recognition (2019, background): “Despite the simple left-to-right nature of speech, several attempts have been made to reach the state-of-the-art performance of a hybrid system with attention based speech recognition models [10, 11, 12].”
Token-wise Training for Attention Based End-to-end Speech Recognition (2019, methodology): “[5] improved MWER training with softmax smoothing for the n-best hypotheses generation process.”
Large Margin Training for Attention Based End-to-End Speech Recognition (2019, methodology): “Results and comparisons of large margin training criteria w/o LM w/ LM Cross Entropy [10] 13.”
Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition (2019, methodology): “Recently, MBR training has been applied in an attention based end-to-end speech recognition system in [22] and further improved in [23].”