Fast, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces
By Frank Zhang, Yongqiang Wang, Xiaohui Zhang, Chunxi Liu, Yatharth Saraf, Geoffry Zweig. 2020
20 citations recorded by the source corpus.
In this work, we first show that on the widely used LibriSpeech benchmark, our transformer-based context-dependent connectionist temporal classification (CTC) system produces state-of-the-art results. We then show that using wordpieces as modeling units combined with CTC training, we can greatly simplify the engineering pipeline compared to conventional frame-based cross-entropy training by excluding all the GMM bootstrapping, decision tree building and force alignment steps, while still achieving very competitive word-error-rate. Additionally, using wordpieces as modeling units can significantly improve runtime efficiency since we can use larger stride without losing accuracy. We further confirm these findings on two internal \emph{VideoASR} datasets: German, which is similar to English as a fusional language, and Turkish, which is an agglutinative language.
Subfields: Artificial Intelligence, Speech & Audio
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Alexei Baevski
- Michael Auli
- Wei-Ning Hsu
- Alexis Conneau
- Luke S. Zettlemoyer
- Graham Neubig
- James R. Glass
- Furu Wei
In Speech & Audio
- Wei-Ning Hsu
- Alexis Conneau
- Alexei Baevski
- Michael Auli
- Abdel-rahman Mohamed
- James R. Glass
- Yu-An Chung
- Yu Zhang
Citations in the corpus
- Unsupervised Speech Recognition (2021, methodology): “We add self-loops that mimic CTC behavior (Zhang et al., 2020a) where blank symbols (silence for the GAN or actual blank symbol for letter models) are mapped to epsilons and consecutive predictions of the same symbol are collapsed.”