Specaugment on Large Scale Datasets

By Daniel S. Park, Yu Zhang, Chung-Cheng Chiu, Youzheng Chen, Bo Li, William Chan, Quoc V. Le, Yonghui Wu. 2019

162 citations recorded by the source corpus.

Recently, SpecAugment, an augmentation scheme for automatic speech recognition that acts directly on the spectrogram of input utterances, has shown to be highly effective in enhancing the performance of end-to-end networks on public datasets. In this paper, we demonstrate its effectiveness on tasks with large scale datasets by investigating its application to the Google Multidomain Dataset (Narayanan et al., 2018). We achieve improvement across all test domains by mixing raw training data augmented with SpecAugment and noise-perturbed training data when training the acoustic model. We also introduce a modification of SpecAugment that adapts the time mask size and/or multiplicity depending on the length of the utterance, which can potentially benefit large scale tasks. By using adaptive masking, we are able to further improve the performance of the Listen, Attend and Spell model on LibriSpeech to 2.2% WER on test-clean and 5.2% WER on test-other.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Yu Zhang
3. Tara N. Sainath
4. Shinji Watanabe
5. Yonghui Wu
6. Dong Yu
7. Chung-Cheng Chiu
8. Quoc V. Le

In Evaluation & Benchmarks

1. Yu Zhang
2. Wei Han
3. Chung-Cheng Chiu
4. Yonghui Wu
5. Bo Li
6. William Chan
7. Ralf Schlüter
8. Mohammad Norouzi

In Speech & Audio

1. Jinyu Li
2. Yu Zhang
3. Tara N. Sainath
4. Yonghui Wu
5. Chung-Cheng Chiu
6. Shinji Watanabe
7. Bo Li
8. Ruoming Pang

In Signal Processing

1. Jinyu Li
2. Yu Zhang
3. Tara N. Sainath
4. Dong Yu
5. Shinji Watanabe
6. Yonghui Wu
7. Chung-Cheng Chiu
8. Bo Li

Citations in the corpus

• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “We use the standard adaptive SpecAugment [42], [43] policy with two frequency masks with size parameter F = 27, and ten time masks with maximum time-mask ratio pS = 0.”
• SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network (2021, positive): “We apply adaptive SpecAugment [4, 46] with two frequency masks of size parameter F = 27 , and ten time masks with maximum time-mask ratio p S = 0 .”
• Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, methodology): “We use an adaptive SpecAugment [9, 10] policy with two frequency masks with mask size parameter F = 27 , and ten time masks with maximum time-mask ratio p S = 0 .”
• Improved Noisy Student Training for Automatic Speech Recognition (2020, methodology): “The choices made for the components of NST are as follows: SpecAugment: For generation 0, we use two frequency masks with mask parameter ( F ) 27, two time masks with mask parameter ( T ) 40, and time warping with warp parameter ( W ) 40 [2].”
• Conformer: Convolution-augmented Transformer for Speech Recognition (2020, methodology): “We use SpecAugment [27, 28] with mask parameter (F = 27), and ten time masks with maximum time-mask ratio (pS = 0.05), where the maximum-size of the time mask is set to pS times the length of the utterance.”
• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, methodology): “The key advantage of a CNN based model is its parameter efficiency; however, the WER achieved by the best CNN model, QuartzNet [4], is still behind the RNN/transformer based models [8, 9, 10, 11].”
• w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training (2021, methodology): “These techniques include SpecAugment [39, 40] for data augmentation, Noisy Student Training [41] for self-training, and language model fusion for decoding.”
• Training Keyword Spotting Models on Non-IID Data with Federated Learning (2020, methodology): “It has been used previously for keyword spotting [40], and has been used to achieve state-of-the-art ASR performance [41].”
• Fast, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces (2020, unknown): “[47] NNLM 2.”
• End-to-End Speech Recognition: A Survey (2023, methodology): “Most data augmentation methods perform data perturbation by exploiting certain dimensions of speech signal variation: speed perturbation [201], [202], vocal tract length perturbation[201],[203],frequencyaxisdistortion[201],sequencenoise injection[204],SpecAugment[205],orsemanticmask[206].”
• Less is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging (2020, methodology): “In this work, we also apply SpecAugment masks [9] using the configuration described in [28], which we find to improve performance over the system in [12].”
• RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions (2020, background): “SpecAugment+VN+RSP obtains a 76% improvement on TTS-Audiobook and a 62% improvement on YT-short.”