Robust Speech Recognition via Large-Scale Weak Supervision

By Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever. International Conference on Machine Learning, 2022

7,824 citations recorded by the source corpus.

We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.

Subfields: Artificial Intelligence, Large Language Models, Speech & Audio, Signal Processing

View Paper: https://arxiv.org/abs/2212.04356

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Yoshua Bengio
  3. Quoc V. Le
  4. Jianfeng Gao
  5. Percy Liang
  6. Jinyu Li
  7. Luke S. Zettlemoyer
  8. Dong Yu

In Large Language Models

  1. Michael Auli
  2. Alexei Baevski
  3. Gabriel Synnaeve
  4. Alexis Conneau
  5. Ankur Bapna
  6. Yossi Adi
  7. Florian Tramèr
  8. Nicholas Carlini

In Speech & Audio

  1. Jinyu Li
  2. Yu Zhang
  3. Wei-Ning Hsu
  4. Tara N. Sainath
  5. Shinji Watanabe
  6. Yonghui Wu
  7. Gabriel Synnaeve
  8. Yossi Adi

In Signal Processing

  1. Jinyu Li
  2. Yu Zhang
  3. Shinji Watanabe
  4. Tara N. Sainath
  5. Dong Yu
  6. Wei-Ning Hsu
  7. Yonghui Wu
  8. Chung-Cheng Chiu

Citations in the corpus