SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
By Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin Dogus Cubuk, Quoc V. Le. 2019
4,125 citations recorded by the source corpus.
We present SpecAugment, a simple data augmentation method for speech recognition. SpecAugment is applied directly to the feature inputs of a neural network (i.e., filter bank coefficients). The augmentation policy consists of warping the features, masking blocks of frequency channels, and masking blocks of time steps. We apply SpecAugment on Listen, Attend and Spell networks for end-to-end speech recognition tasks. We achieve state-of-the-art performance on the LibriSpeech 960h and Swichboard 300h tasks, outperforming all prior work. On LibriSpeech, we achieve 6.8% WER on test-other without the use of a language model, and 5.8% WER with shallow fusion with a language model. This compares to the previous state-of-the-art hybrid system of 7.5% WER. For Switchboard, we achieve 7.2%/14.6% on the Switchboard/CallHome portion of the Hub5'00 test set without the use of a language model, and 6.8%/14.1% with shallow fusion, which compares to the previous state-of-the-art hybrid system at 8.3%/17.3% WER.
Subfields: Artificial Intelligence, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- James R. Glass
In Speech & Audio
- Jinyu Li
- Wei-Ning Hsu
- Yu Zhang
- Shinji Watanabe
- James R. Glass
- Abdel-rahman Mohamed
- Hung-yi Lee
- Tara N. Sainath
In Signal Processing
Citations in the corpus
- Unsupervised Data Augmentation for Consistency Training (2019, background): “In recent years, there have been significant advancements on the design of data augmentations for NLP [12], vision [10, 11] and speech [13, 14] in supervised settings.”
- Self-Supervised MultiModal Versatile Networks (2020, neutral): “Mildly jittering with SpecAugment [64] was not beneficial, and more aggressive augmentations were detrimental; in contrast with the findings of [65] where SpecAugment helped, presumably due to training on a relatively small dataset.”
- Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Data augmentation is a necessary method for achieving state-of-the-art performance [36, 17, 6, 48, 11, 30].”
- Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, methodology): “5 WER (Park et al., 2019), and our Imputer model achieves 11.”
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, positive): “All models use SpecAugment (Park et al., 2019) with an LD policy.”
- RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation (2019, methodology): “Our best hybrid model even outperforms the endto-end attention model with SpecAugment [10] by 8% relative WER on test-clean and by 13.”
- Perceiver: General Perception with Iterative Attention (2021, methodology): “For spectrograms we use also specaugment (Park et al., 2019).”
- FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech (2022, background): “SpecAugment [23], speed perturbation [24], simulated reverberation [25], etc.”
- MAESTRO: Matched Speech Text Representations through Modality Matching (2022, methodology): “This is the RNN-T loss applied over the masked, resampled text embeddings with masking in frequency and time domain similar to SpecAugment [28].”
- BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “The pre-training method used in this work is based on wav2vec 2.0 [12], while the self-training method is based on noisy student training [41] using SpecAugment [42], [43].”
- SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network (2021, positive): “The same SpecAugment policy as the 100M model is used to train this model, while exponential moving averages are not used.”
- MLS: A Large-Scale Multilingual Dataset for Speech Research (2020, methodology): “SpecAugment[24] is used for data augmentation in training: there are two frequency masks, and ten time masks with maximum time mask ratio of p = 0.1; frequency and time mask parameters are tuned separately for each model; time warping is not used.”
- Self-Training and Pre-Training are Complementary for Speech Recognition (2020, neutral): “960h labeled Supervised SpecAugment [1] - - 2.”
- Rethinking Evaluation in ASR: Are Our Models Robust Enough? (2020, methodology): “SpecAugment [29] is used for data augmentation in training: there are two frequency masks, and ten time masks with maximum time mask ratio of p = 0.”
- Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, methodology): “We use an adaptive SpecAugment [9, 10] policy with two frequency masks with mask size parameter F = 27 , and ten time masks with maximum time-mask ratio p S = 0 .”