MLS: A Large-Scale Multilingual Dataset for Speech Research

By Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, Ronan Collobert. 2020

803 citations recorded by the source corpus.

This paper introduces Multilingual LibriSpeech (MLS) dataset, a large multilingual corpus suitable for speech research. The dataset is derived from read audiobooks from LibriVox and consists of 8 languages, including about 44.5K hours of English and a total of about 6K hours for other languages. Additionally, we provide Language Models (LM) and baseline Automatic Speech Recognition (ASR) models and for all the languages in our dataset. We believe such a large transcribed dataset will open new avenues in ASR and Text-To-Speech (TTS) research. The dataset will be made freely available for anyone at this http URL.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Speech & Audio

In Signal Processing

Citations in the corpus

Robust Speech Recognition via Large-Scale Weak Supervision (2022, methodology): “Toshniwal et al. (2018) demonstrated jointly training a modern deep learning speech recognition system on several languages with a single model, and Pratap et al. (2020a) scaled this line of work significantly to 50 languages with a billion-parameter model.”
FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech (2022, methodology): “train on speech data, incorporating textual and speech-text data into pre-training allows for transfer learning across the two modalities [13].”
MAESTRO: Matched Speech Text Representations through Modality Matching (2022, methodology): “Our transcribed data draws from VoxPopuli transcribed dataset (VP-P), MLS and Babel, following [11].”
mSLAM: Massively multilingual joint pre-training for speech and text (2022, methodology): “1 release of Common Voice (Ardila et al., 2019), 50k hours of read books data in eight European languages from Multilingual LibriSpeech (Pratap et al., 2020) and 1k hours of telephonic conversation data spanning 17 African and Asian languages from BABEL (Gales et al., 2014).”
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale (2021, methodology): “Multilingual Librispeech (MLS) contains data in eight European languages totaling around 50K hours of data [20].”
The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage (2021, methodology): “This work is a first step towards creating a speech dataset that covers all speakers and environments (not just English audiobooks [22, 25, 26]) for open use [27].”
Unsupervised Speech Recognition (2021, methodology): “3 Performance on non-English languages To get a sense of how well the method works on non-English data, we experiment on six languages of the multilingual Librispeech corpus (MLS; Pratap et al. 2020).”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, background): “…et al., 2018; Schneider et al., 2019; Baevski et al., 2020; Conneau et al., 2020; Wu et al., 2020; Nguyen et al., 2020), with semi-supervised learning (self-training) (Kahn et al., 2020a; Pino et al., 2020; Zhang et al., 2020b; Xu et al., 2020) or a combination of both methods (Xu et al., 2020).”
Audiobox: Unified Audio Generation with Natural Language Prompts (2023, background): “…generative models (Wang et al., 2023a; Le et al., 2023; Shen et al., 2023) are trained mostly on audiobooks (Zen et al., 2019; Kahn et al., 2019; Pratap et al., 2020), which lacks diversity compared to truly in-the-wild data such as AudioSet (Gemmeke et al., 2017) in terms of expressivity…”
Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context (2023, background): “In recent years, there are also large-scale corpora like GigaSpeech [9], People’s Speech [10] and MLS [11].”
RepCodec: A Speech Representation Codec for Speech Tokenization (2023, methodology): “For multilingual experiments, we further incorporate 100h subsets from MLS (Pratap et al., 2020) for French and Spanish.”
AudioPaLM: A Large Language Model That Can Speak and Listen (2023, positive): “In both cases we train on Multilingual LibriSpeech [Pratap et al., 2020] and the voice conditioning is supplied as a 3-second long voice sample, represented as both audio tokens and SoundStream tokens.”
NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (2023, methodology): “Training Dataset To train the neural audio codec and the diffusion model, we use the English subset of Multilingual LibriSpeech (MLS) [41] as the training data, which contains 44K hours of transcribed speech data derived from LibriVox audiobooks.”
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, positive): “In addition, we evaluated the model in the test subset of Multilingual LibriSpeech (MLS) [33] us-ing all languages.”
A Survey on Neural Speech Synthesis (2021, neutral): “1 German India Corpus [103] 39 253 48 Multilingual M-AILABS [85] 1000 / 16 Multilingual MLS [272] 51K 6K 16 Multilingual CSS10 [258] 140 1 22.”
Unsupervised Cross-lingual Representation Learning for Speech Recognition (2020, methodology): “The Multilingual LibriSpeech dataset [36] is a large corpus derived from read audiobooks of Librivox and consists of 8 languages: Dutch (du), English (en), French (fr), German (de), Italian (it), Polish (pl), Portuguese (pt), Spanish (es).”
WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition (2021, background): “To close the gap between industrial system and academic research, we notice that several large-scale multi-domain English corpora, including The People’s Speech [24], MLS [25] and GigaSpeech [26], are made available recently.”