Multilingual Speech Recognition with Corpus Relatedness Sampling

By Xinjian Li, Siddharth Dalmia, Alan W. Black, Florian Metze. 2019

17 citations recorded by the source corpus.

Multilingual acoustic models have been successfully applied to low-resource speech recognition. Most existing works have combined many small corpora together and pretrained a multilingual model by sampling from each corpus uniformly. The model is eventually fine-tuned on each target corpus. This approach, however, fails to exploit the relatedness and similarity among corpora in the training set. For example, the target corpus might benefit more from a corpus in the same domain or a corpus from a close language. In this work, we propose a simple but useful sampling strategy to take advantage of this relatedness. We first compute the corpus-level embeddings and estimate the similarity between each corpus. Next, we start training the multilingual model with uniform-sampling from each corpus at first, then we gradually increase the probability to sample from related corpora based on its similarity with the target corpus. Finally, the model would be fine-tuned automatically on the target corpus. Our sampling strategy outperforms the baseline multilingual model on 16 low-resource tasks. Additionally, we demonstrate that our corpus embeddings capture the language and domain information of each corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised Pretraining Transfers Well Across Languages (2020, background): “The sampling of the languages during the pre-training can focus on languages related to the targeted language [11].”
Universal Phone Recognition with a Multilingual Allophone System (2020, background): “The second group, private phoneme models, treats phonemes from each language as completely different classes performs phoneme classification separately for each language [11, 2, 12].”