Model Unit Exploration for Sequence-to-Sequence Speech Recognition

By Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen. 2019

37 citations recorded by the source corpus.

In conventional speech recognition, phoneme-based models outperform grapheme-based models for non-phonetic languages such as English. The performance gap between the two typically reduces as the amount of training data is increased. In this work, we examine the impact of the choice of modeling unit for attention-based encoder-decoder models. We conduct experiments on the LibriSpeech 100hr, 460hr, and 960hr tasks, using various target units (phoneme, grapheme, and word-piece); across all tasks, we find that grapheme or word-piece models consistently outperform phoneme-based models, even though they are evaluated without a lexicon or an external language model. We also investigate model complementarity: we find that we can improve WERs by up to 9% relative by rescoring N-best lists generated from a strong word-piece based baseline with either the phoneme or the grapheme model. Rescoring an N-best list generated by the phonemic system, however, provides limited improvements. Further analysis shows that the word-piece-based models produce more diverse N-best hypotheses, and thus lower oracle WERs, than phonemic models.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Speech & Audio

Citations in the corpus

Multitask Training with Text Data for End-to-End Speech Recognition (2020, methodology): “The E2E models in our experiments are 8-layer listen, attend, and spell (LAS) models based on the large model from [32].”
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, methodology): “For LibriSpeech, we use a two-layer RNN with embedding dimension 1024 used in [25] for the LM, which is trained on the LibriSpeech LM corpus.”
Effectiveness of Self-Supervised Pre-Training for ASR (2020, methodology): “All results use 4-gram language models, except [37, 38].”
Recent Advances in End-to-End Automatic Speech Recognition (2021, background): “While most units (characters [74, 80], word-piece [37, 101, 244, 308, 322], words [7, 154, 259], or even phrases [67]) are purely derived based on text and word-piece unit is the dominating one, there are also studies working on phoneme units [112, 234, 293, 345].”
Advancing Sequence-to-Sequence Based Speech Recognition (2019, result): “Optimizing the sub-sampling rate for each representation unit, we found that, in contrast to other papers, larger BPE units are not beneficial if external language model is also used [11, 39].”