Massively Multilingual Adversarial Speech Recognition

By Oliver Adams, Matthew Wiesner, Shinji Watanabe, David Yarowsky. 2019

83 citations recorded by the source corpus.

We report on adaptation of multilingual end-to-end speech recognition models trained on as many as 100 languages. Our findings shed light on the relative importance of similarity between the target and pretraining languages along the dimensions of phonetics, phonology, language family, geographical location, and orthography. In this context, experiments demonstrate the effectiveness of two additional pretraining objectives in encouraging language-independent encoder representations: a context-independent phoneme objective paired with a language-adversarial classification objective.

Subfields: Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Speech & Audio

Citations in the corpus

Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters (2020, background): “Prior work has leveraged as many as 100 languages simultaneously to learn language agnostic features [12].”
Unsupervised Pretraining Transfers Well Across Languages (2020, background): “Another approach is to encourage a language-independent encoder with an adversarial loss [13].”
End-to-End Speech Recognition: A Survey (2023, unknown): “Pretrainingapproachesutilizinguntranscribedaudio, large-scalesemi-superviseddataand/ormultilingualdata[149], [150],[151],[152],[153],[154],[155],[156],[157],[158], [159],[160]woulddeserveaself-containedsurveyandtheyare applicableforhybridDNN/HMMandE2Eapproacheslikewise…”
Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training (2020, methodology): “Domain adversarial learning with GRL is used for ASR in [12, 13].”
Scaling End-to-End Models for Large-Scale Multilingual ASR (2021, background): “Prior work explored as many as 50 [19] to 100 [21] languages.”
Pretraining by Backtranslation for End-to-End ASR in Low-Resource Settings (2018, background): “And while multilingual training often provides more significant improvements in performance, this approach also requires additional transcribed speech, preferably from similar languages [11, 15].”
MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible (2019, background): “, 2016), automatic speech recognition (Schultz and Schlippe, 2014; Adams et al., 2019), lexical disambiguation (Navigli and Ponzetto, 2010; Sérasset, 2015), and computational dialectology (Christodoulopoulos and Steedman, 2015).”
Analyzing ASR Pretraining for Low-Resource Speech-to-Text Translation (2019, background): “Although our main focus is monolingual pretraining, we also looked briefly at multilingual pretraining, inspired by recent work on multilingual ASR [29, 30] and evidence that multilingual pretraining followed by fine-tuning on a distinct target language can improve ASR on the target language [11, 31”