VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

By Changhan Wang, Morgane Rivière, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux. 2021

744 citations recorded by the source corpus.

We introduce VoxPopuli, a large-scale multilingual corpus providing 400K hours of unlabeled speech data in 23 languages. It is the largest open data to date for unsupervised representation learning as well as semi-supervised learning. VoxPopuli also contains 1.8K hours of transcribed speeches in 15 languages and their aligned oral interpretations into 15 target languages totaling 17.3K hours. We provide speech recognition (ASR) baselines and validate the versatility of VoxPopuli unlabeled data in semi-supervised ASR and speech-to-text translation under challenging out-of-domain settings. The corpus is available at https://github.com/facebookresearch/voxpopuli.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Evaluation & Benchmarks, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Evaluation & Benchmarks

In Speech & Audio

Citations in the corpus

Robust Speech Recognition via Large-Scale Weak Supervision (2022, methodology): “• VoxPopuli (Wang et al., 2021): We used the get asr data.py script from the official repository to collect the ASR data in 16 languages, including English.”
MAESTRO: Matched Speech Text Representations through Modality Matching (2022, methodology): “Multilingual versions of Maestro are fine-tuned and evaluated on the 14-language VoxPopuli [33] multilingual ASR task using grapheme targets [11].”
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale (2021, methodology): “We pretrain our models on a total of 436K hours of publicly available data from the following sources: VoxPopuli (VP-400K) comprises a total of 372K hours of data in 23 European languages of parliamentary speech from the European parliament [19].”
Unispeech-Sat: Universal Speech Representation Learning With Speaker Aware Pre-Training (2021, positive): “…podcasts and YouTube, covering both read and spontaneous speaking styles, and a variety of topics, such as arts, science, sports, etc. (2) 24K hours VoxPopuli data [18]), which from European Parliament (EP) event recordings including plenary sessions, committee meetings and other events.”
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training (2021, methodology): “We use the LMs provided in [8] for LS, SB, TD, CV, WS, and that from [35] for VP.”
AudioPaLM: A Large Language Model That Can Speak and Listen (2023, neutral): “• VoxPopuli [Wang et al., 2021] contains speeches from the European Parliament together with their transcripts – which can be used for speech recognition (ASR) tasks – as well as spoken translations from parliamentary interpreters – which can be used for speech translation (S2ST) tasks.”
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling (2023, neutral): “Though researchers have put much effort into constructing speech-to-speech translation corpora, such as Voxvopule [Wang et al., 2021], CVSS [Jia et al., 2022b], and SpeechMatrix [Duquenne et al., 2022], they are either synthesized from text or mined from multilingual speech corpora thus can not…”
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, methodology): “Since our focus is English-only audio, we use 24k hours of English data in VoxPopuli for pre-training.”
Direct Speech-to-Speech Translation With Discrete Units (2021, methodology): “With the recent release of large-scale S2S dataset [37], we plan to investigate the proposed framework with real data in the future.”
CVSS Corpus and Massively Multilingual Speech-to-Speech Translation (2022, positive): “As a comparison, translation is typically verbatim and exact, while interpretation is typically summarizing and often drops less important details; there is also more linguistic variation and disﬂuencies in interpretation (He et al., 2016; Shimizu et al., 2013; Wang et al., 2021a).”
WhisperX: Time-Accurate Speech Transcription of Long-Form Audio (2023, methodology): “Base 960h and Large 960h models were trained on Lib-rispeech [27] data, whereas the VoxPopuli model was trained on the Voxpopuli [28] corpus.”
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation (2022, positive): “We evaluate the proposed model on Europarl-ST [15] and Vox-Populi [5] S2ST datasets.”