Training Neural Speech Recognition Systems with Synthetic Speech Augmentation

By Jason Li, Ravi Teja Gadde, Boris Ginsburg, Vitaly Lavrukhin. 2018

61 citations recorded by the source corpus.

Building an accurate automatic speech recognition (ASR) system requires a large dataset that contains many hours of labeled speech samples produced by a diverse set of speakers. The lack of such open free datasets is one of the main issues preventing advancements in ASR research. To address this problem, we propose to augment a natural speech dataset with synthetic speech. We train very large end-to-end neural speech recognition models using the LibriSpeech dataset augmented with synthetic speech. These new models achieve state of the art Word Error Rate (WER) for character-level based models without an external language model.

Subfields: Artificial Intelligence, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

Citations in the corpus

You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation (2020, background): “Apart from human-computer interaction applications, such high-quality artificial speech can be used as a data augmentation for ASR [8].”
Generating Diverse and Natural Text-to-Speech Samples Using a Quantized Fine-Grained VAE and Autoregressive Prosody Prior (2020, methodology): “Similar to [28], synthesizing speech using a baseline multispeaker Tacotron 2 model results in a significantly degraded WER of 20.”
Speech Recognition with Augmented Synthesized Speech (2019, result): “We draw particular comparison with [3], prior work exploring speech synthesis based data augmentation on LIBRISPEECH.”
Improving Speech Recognition Using Consistent Predictions on Synthesized Speech (2020, unknown): “This can be con-ﬁrmed from previous work [6], where ASR models trained mostly on TTS synthesized data fail to generalize to real speech utterances.”
Synthetic Data for Deep Learning (2019, background): “For example, while this survey does not yet have a section devoted to sound and speech processing, works that use synthetic data in this domain are already beginning to appear [353, 510].”
Self-Supervised Speech Representation Learning: A Review (2022, methodology): “straightforward approach is to simply use multi-speaker TTS to generate the waveform with various acoustic variations [314], [315], [316], [317], [318].”
Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems (2019, methodology): “While related work covering independent TTS systems [10, 11] uses additional data, our TTS system is only trained on the ASR corpus itself.”
Injecting Text in Self-Supervised Speech Pretraining (2021, background): “A major challenge in using TTS for ASR data augmentation is encouraging effective generalization from synthetic to real speech [17, 16, 42].”