A Comparative Study on End-to-End Speech to Text Translation

By Parnia Bahar, Tobias Bieschke, Hermann Ney. 2019

95 citations recorded by the source corpus.

Recent advances in deep learning show that end-to-end speech to text translation model is a promising approach to direct the speech translation field. In this work, we provide an overview of different end-to-end architectures, as well as the usage of an auxiliary connectionist temporal classification (CTC) loss for better convergence. We also investigate on pre-training variants such as initializing different components of a model using pretrained models, and their impact on the final performance, which gives boosts up to 4% in Bleu and 5% in Ter. Our experiments are performed on 270h IWSLT TED-talks En→De, and 100h LibriSpeech Audio-books En→Fr. We also show improvements over the current end-to-end state-of-the-art systems on both tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Juan Pino
2. Changhan Wang
3. Wei-Ning Hsu
4. Jiatao Gu
5. Yossi Adi
6. Alexis Conneau
7. Ann Lee
8. Michael Auli

Citations in the corpus

• Direct Speech-to-Speech Translation With Discrete Units (2021, background): “, 2017), model pre-training (Bahar et al., 2019; Li et al., 2021) or data augmentation (Jia et al.”
• Multilingual Speech Translation from Efficient Finetuning of Pretrained Models (2021, methodology): “Multi-task learning has been shown as an effective approach to improve the performance of the speech translation task using other related tasks, such as MT and ASR (Weiss et al., 2017; Anastasopoulos and Chiang, 2018; Bahar et al., 2019; Tang et al., 2021a,b).”
• A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks (2020, background): “Another direction is to leverage text data directly during training through multitask learning [18].”
• ESPnet-ST: All-in-One Speech Translation Toolkit (2020, methodology): “We used the clean 100-hours of speech data and augmented translation references with Google Translate for the training set (B´erard et al., 2018; Liu et al., 2019; Bahar et al., 2019a,b).”