XTREME-S: Evaluating Cross-lingual Speech Representations
By Alexis Conneau, Ankur Bapna, Yu Zhang, Min Ma, Patrick von Platen, Anton Lozhkov, Colin Cherry, Ye Jia, Clara Rivera, Mihir Kale and 9 more. 2022
27 citations recorded by the source corpus.
We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual speech representations in many languages. XTREME-S covers four task families: speech recognition, classification, speech-to-text translation and retrieval. Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in"universal"speech representation learning. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines using XLS-R and mSLAM on all downstream tasks. We motivate the design choices and detail how to use the benchmark. Datasets and fine-tuning scripts are made easily accessible at https://hf.co/datasets/google/xtreme_s.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Zalán Borsos
- Neil Zeghidour
- Evgeny Kharitonov
- Damien Vincent
- Matthew Sharifi
- Marco Tagliasacchi
- Raphaël Marinier
- David Grangier
Citations in the corpus
- AudioLM: A Language Modeling Approach to Audio Generation (2022, background): “While this model can be fine-tuned for discriminative tasks such as speech recognition or speech-to-text translation [48], AudioLM rather leverages the representations of the pre-trained w2v-BERT to model long-”