Analyzing ASR Pretraining for Low-Resource Speech-to-Text Translation

By Mihaela C. Stoian, Sameer Bansal, Sharon Goldwater. 2019

75 citations recorded by the source corpus.

Previous work has shown that for low-resource source languages, automatic speech-to-text translation (AST) can be improved by pre-training an end-to-end model on automatic speech recognition (ASR) data from a high-resource language. However, it is not clear what factors—e.g., language relatedness or size of the pretraining data— yield the biggest improvements, or whether pretraining can be effectively combined with other methods such as data augmentation. Here, we experiment with pretraining on datasets of varying sizes, including languages related and unrelated to the AST source language. We find that the best predictor of final AST performance is the word error rate of the pretrained ASR model, and that differences in ASR/AST performance correlate with how phonetic information is encoded in the later RNN layers of our model. We also show that pretraining and data augmentation yield complementary benefits for AST.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Self-Training for End-to-End Speech Translation (2020, neutral): “Data scarcity has been addressed in previous work with data augmentation [1, 2], multi-task training [3, 4], pre-training [5, 6] or multilingual speech translation [7, 8, 9].”
Learning Shared Semantic Space for Speech-to-Text Translation (2021, unknown): “Standard training techniques such as pretraining (Weiss et al., 2017; B´erard et al., 2018; Bansal et al., 2019; Stoian et al., 2020; Wang et al., 2020a; Pino et al., 2020), multi-task training (Vydana et al., 2021; Le et al., 2020; Tang et al., 2021), meta-learning (Indurthi et al., 2020), and…”
Self-Supervised Representations Improve End-to-End Speech Translation (2020, methodology): “Transferring the parameters learned on a higher-resource ASR task has been shown to be an effective way to improve the performance and ameliorate the training of low-resource ST [16, 17, 18], thus we also study the interactions with selfsupervised representations and whether we can effectively combi”
Large-Scale Self- and Semi-Supervised Learning for Speech Translation (2021, background): “Similarly for speech translation, there has been work on combating data scarcity, namely multitask learning [11, 12, 13], pretraining on ASR data [12, 14, 15, 16], data augmentation [17, 18, 19, 20], self-supervised pretraining [21, 22], selftraining [23] or multilingual speech translation [24, 25, ”