CoVoST 2 and Massively Multilingual Speech-to-Text Translation

By Changhan Wang, Anne Wu, Juan Pino. 2020

181 citations recorded by the source corpus.

Subfields: Machine Translation, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Machine Translation

In Speech & Audio

Citations in the corpus

Robust Speech Recognition via Large-Scale Weak Supervision (2022, methodology): “We attribute this to the 68,000 hours of X → en translation data for these languages in our pre-training dataset which, although noisy, is vastly larger than the 861 hours of training data for X → en translation in CoVoST2.”
FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech (2022, background): “Such recent progress has been made possible with the release of both large-scale pre-training and evaluation datasets like Multilingual LibriSpeech [13], VoxPopuli [14], CoVoST2 [15], CommonVoice [16], and the re-use of existing datasets like BABEL [17].”
MAESTRO: Matched Speech Text Representations through Modality Matching (2022, methodology): “We explore pre-training Maestro on both monolingual (English) and multilinugal scenarios as well as ASR and translation tasks.”
mSLAM: Massively multilingual joint pre-training for speech and text (2022, methodology): “On several downstream speech understanding tasks, including CoVoST-2 21→En speech translation (Wang et al., 2021b), Fleurs speech language identification (Section 4.2) and Minds-14 speech intent classification (Gerz et al., 2021), mSLAM demonstrates significant quality improvements over equivalent…”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, methodology): “…and application of unsupervised speech pre-training methods (Oord et al., 2018; Schneider et al., 2019; Baevski et al., 2020; Conneau et al., 2020; Wu et al., 2020; Nguyen et al., 2020), with semi-supervised learning (self-training) (Kahn et al., 2020a; Pino et al., 2020; Zhang et al., 2020b; Xu…”
Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq (2020, methodology): “We evaluate FAIRSEQ S2T models on English ASR benchmark—LibriSpeech (Panayotov et al., 2015), as well as multilingual ST benchmarks—MuST-C (Di Gangi et al., 2019a) and CoVoST 2 (Wang et al., 2020c).”
AudioPaLM: A Large Language Model That Can Speak and Listen (2023, neutral): “• CoVoST2 [Wang et al., 2020] is a speech-to-text dataset mapping speech in 21 languages to English text.”
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation (2022, neutral): “# samples source (hrs) target (hrs) S2T, Es-En CoVoST-2 [35] 78.”
Multilingual Speech Translation from Efficient Finetuning of Pretrained Models (2021, methodology): “Previous SOTAs: We compare to the best end-to-end (E2E) model from previous literature (Wang et al., 2020b; Iranzo-S´anchez et al., 2020) on each translation direction, which is usually the best-performing multilingual model trained with parallel data from all directions (both X-En and En-X) and…”
Self-Supervised Speech Representation Learning: A Review (2022, methodology): “CoVoST-2 is a multilingual speech translation benchmark based on CV.”
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages (2023, methodology): “• Automatic Speech Translation (AST): We test AST performance on CoVoST 2 [18].”
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation (2022, methodology): “For a fair comparison, we use the benchmark CVSS-C dataset (Jia et al., 2022), which is derived from the CoVoST 2 (Wang et al., 2020b) speech-to-text translation corpus by synthesizing the translation text into speech using a single-speaker TTS system.”
Large-Scale Self- and Semi-Supervised Learning for Speech Translation (2021, methodology): “1 BLEU points better than the baseline of [27] (row 1) and 8.”
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training (2021, methodology): “Speech translation: CoVoST 2 (Wang et al., 2021a) is a large-scale multilingual speech translation corpus covering translations from 21 languages into English and from English into 15 languages.”
XTREME-S: Evaluating Cross-lingual Speech Representations (2022, background): “For speech translation, we use all the 21 language pairs into English from the CoVoST-2 dataset.”
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data (2022, methodology): “CoVoST-2 [31] is a large-scale multilingual Speech Translation corpus based on CommonVoice [32], containing about 430 hours of reading-style speech from the Internet.”
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT (2023, methodology): “For the S2TT task, we employ the commonly used BSTC (Zhang et al., 2021) and CoVOST 2 (Wang et al., 2020) datasets.”