Low Resource Multi-modal Data Augmentation for End-to-end ASR
By Adithya Renduchintala, Shuoyang Ding, Matthew Wiesner, Shinji Watanabe. 2018
29 citations recorded by the source corpus.
We explore training attention-based encoder-decoder ASR for low-resource languages and present techniques that result in a 50% relative improvement in character error rate compared to a standard baseline. The performance of encoder-decoder ASR systems depends on having sufficient target-side text to train the attention and decoder networks. The lack of such data in low-resource contexts results in severely degraded performance. In this paper we present a data augmentation scheme tailored for low-resource ASR in diverse languages. Across 3 test languages, our approach resulted in a 20% average relative improvement over a baseline text-based augmentation technique. We further compare the performance of our monolingual text-based data augmentation to speech-based data augmentation from nearby languages and find that this gives a further 20-30% relative reduction in character error rate.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Jinyu Li
- Shinji Watanabe
- Eduard H. Hovy
- Changhan Wang
- Varun Prashant Gangal
- Juan Pino
- Teruko Mitamura
- Soroush Vosoughi
Citations in the corpus
- A Survey of Data Augmentation Approaches for NLP (2021, methodology): “Wiesner et al. (2018) and Renduchintala et al. (2018) propose DA methods for end-to-end automatic speech recognition (ASR).”
- An open-source end-to-end ASR system for Brazilian Portuguese using DNNs built from newly assembled corpora (2020, methodology): “While [14] consists on training an extra encoder for its augmentation scheme, and [15] uses a transformer topology with more than 200 million parameters, our DeepSpeech 2 based model has fewer parameters (42 millions) and does not require extra parameters for pretraining.”
- A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks (2020, methodology): “[19] proposed multi-modal data augmentation to jointly train text and speech for ASR, but little gain is observed if no LM is used.”
- An Attention-Based Joint Acoustic and Text on-Device End-To-End Model (2020, background): “A main difference is that [22] shares attention and decoder parameters across all modes, only changing the parameters of the input passed to the encoder depending on the input source.”
- Recent Advances in End-to-End Automatic Speech Recognition (2021, methodology): “In [228], a multi-modal data augmentation method was proposed by having two separate encoders: one for acoustic input and the other for text input, sharing the same attention”
- Pretraining by Backtranslation for End-to-End ASR in Low-Resource Settings (2018, methodology): “The same baseline model trained as in [16] using an”