Toward Domain-Invariant Speech Recognition via Large Scale Training

By Arun Narayanan, Ananya Misra, Khe Chai Sim, Golan Pundak, Anshuman Tripathi, Mohamed G. Elfeky, Parisa Haghani, Trevor Strohman, Michiel Bacchiani. 2018

114 citations recorded by the source corpus.

Current state-of-the-art automatic speech recognition systems are trained to work in specific ‘domains’, defined based on factors like application, sampling rate and codec. When such recognizers are used in conditions that do not match the training domain, performance significantly drops. This work explores the idea of building a single domain-invariant model for varied use-cases by combining large scale training data from multiple application domains. Our final system is trained using 162,000 hours of speech. Additionally, each utterance is artificially distorted during training to simulate effects like background noise, codec distortion, and sampling rates. Our results show that, even at such a scale, a model thus trained works almost as well as those fine-tuned to specific subsets: A single model can be robust to multiple application domains, and variations like codecs and noise. More importantly, such models generalize better to unseen conditions and allow for rapid adaptation – we show that by using as little as 10 hours of data from a new domain, an adapted domain-invariant model can match performance of a domain-specific model trained from scratch using 70 times as much data. We also highlight some of the limitations of such models and areas that need addressing in future work.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Yu Zhang
3. Tara N. Sainath
4. Yonghui Wu
5. Dong Yu
6. Bo Li
7. Quoc V. Le
8. Chung-Cheng Chiu

In Speech & Audio

1. Jinyu Li
2. Yu Zhang
3. Tara N. Sainath
4. Yonghui Wu
5. Chung-Cheng Chiu
6. Bo Li
7. Ruoming Pang
8. Rohit Prabhavalkar

In Signal Processing

1. Jinyu Li
2. Tara N. Sainath
3. Yu Zhang
4. Yonghui Wu
5. Dong Yu
6. Chung-Cheng Chiu
7. Bo Li
8. Rohit Prabhavalkar

Citations in the corpus

• Robust Speech Recognition via Large-Scale Weak Supervision (2022, background): “As demonstrated by Narayanan et al. (2018), Likhomanenko et al. (2020), and Chan et al. (2021) speech recognition systems that are pre-trained in a supervised fashion across many datasets/domains exhibit higher robustness and generalize much more effectively to held-out datasets than models trained…”
• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, background): “Besides the obvious advantage of having a very large amount of audio available, these YouTube-based datasets have an extremely wide range of sub-domains [54], [69], plotted in Fig.”
• SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network (2021, neutral): “Narayanan et al. [13] mixed several proprietary commercial datasets, totally more than 160k hours of training data to train one neural network as well.”
• Specaugment on Large Scale Datasets (2019, methodology): “In this paper, we address this question by applying SpecAugment to the Google Multidomain Dataset introduced in [2].”
• Universal ASR: Unify and Improve Streaming ASR with Full-context Modeling (2020, background): “For MultiDomain, we report our evaluation results on Voice Search test set and compare with our reproduced baselines.”
• Deep Learning for Audio Signal Processing (2019, methodology): “data and then adapted to a low-resource language [75] or domains [76].”
• An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models (2019, background): “Automatic speech recognition (ASR) systems are trained with thousands of hours of speech data from many users to be robust against speaker variability and achieve state-of-the-art performance [1, 2, 3].”
• Recognizing Long-Form Speech Using Streaming End-to-End Models (2019, methodology): “Additionally, we generate both 8 kHz and 16 kHz versions of the training data to reduce acoustic mismatch due to sampling rates [19, 34].”
• RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions (2020, unknown): “be expected in conventional models [20].”
• A Comparison of End-to-End Models for Long-Form Speech Recognition (2019, background): “YouTube videos cover a wide variety of different domains [20], and have a wide range of length distributions, making it an ideal test bed for this long-form transcription study.”