CoVoST 2: A Massively Multilingual Speech-to-Text Translation Corpus

By Changhan Wang, Anne Wu, Juan Pino. 2020

99 citations recorded by the source corpus.

Speech translation has recently become an increasingly popular topic of research, partly due to the development of benchmark datasets. Nevertheless, current datasets cover a limited number of languages. With the aim to foster research in massive multilingual speech translation and speech translation for low resource language pairs, we release CoVoST 2, a large-scale multilingual speech translation corpus covering translations from 21 languages into English and from English into 15 languages. This represents the largest open dataset available to date from total volume and language coverage perspective. Data sanity checks provide evidence about the quality of the data, which is released under CC0 license. We also provide extensive speech recognition, bilingual and multilingual machine translation and speech translation baselines.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Juan Pino
2. Changhan Wang
3. Jiatao Gu
4. Wei-Ning Hsu
5. Mingxuan Wang
6. Ann Lee
7. Ye Jia
8. Peng-Jen Chen

Citations in the corpus

• Textless Speech-to-Speech Translation on Real Data (2021, positive): “All experiments are conducted using fairseq (Ott et al., 2019; Wang et al., 2020a, 2021b) 4 .”
• The Multilingual TEDx Corpus for Speech Recognition and Translation (2021, methodology): “Collecting data in a variety of genres is critical for building generalizable models and assessing real-world performance of ST systems, but many ST corpora consist only of read speech [9, 11, 13, 14].”
• CoBERT: Self-Supervised Speech Representation Learning Through Code Representation Learning (2022, methodology): “We use the CoVoST2 en→de dataset [21] for evaluation.”
• Translatotron 2: Robust direct speech-to-speech translation (2021, methodology): “We trained both a Translatotron 2 model and a Translatotron model on the 4 high-resource languages from the CoVoST 2 corpus (Wang et al., 2020a), using TTS synthesized target speech in a single female speaker’s voice.”
• SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities (2022, unknown): “We use the CoVoST2 En Ñ De (Wang et al., 2020) dataset (CC0 Licensed) with their official train, validation, and test splits while removing all the samples containing "REMOVE", resulting in 425.8, 25.9 and 24.5 hours respectively.”
• Translatotron 2: High-quality direct speech-to-speech translation with voice preservation (2021, methodology): “…outperformed the cascade of ASR and MT (Weiss et al., 2017; Jia et al., 2019a; Di Gangi et al., 2019; McCarthy et al., 2020; Ansari et al., 2020; Wang et al., 2021b; Anastasopoulos et al., 2021), which makes the Proceedings of the 39 th International Conference on Machine Learning , Baltimore,…”