Speech Translation and the End-to-End Promise: Taking Stock of Where We Are

By Matthias Sperber, Matthias Paulik. 2020

126 citations recorded by the source corpus.

Over its three decade history, speech translation has experienced several shifts in its primary research themes; moving from loosely coupled cascades of speech recognition and machine translation, to exploring questions of tight coupling, and finally to end-to-end models that have recently attracted much attention. This paper provides a brief survey of these developments, along with a discussion of the main challenges of traditional approaches which stem from committing to intermediate representations from the speech recognizer, and from training cascaded models separately towards different objectives. Recent end-to-end modeling techniques promise a principled way of overcoming these issues by allowing joint training of all model components and removing the need for explicit intermediate representations. However, a closer look reveals that many end-to-end models fall short of solving these issues, due to compromises made to address data scarcity. This paper provides a unifying categorization and nomenclature that covers both traditional and recent approaches and that may help researchers by highlighting both trade-offs and open research questions.

Subfields: Artificial Intelligence, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pascale Fung
2. Andrea Madotto
3. Wenliang Dai
4. Nayeon Lee
5. Yan Xu
6. Etsuko Ishii
7. Ziwei Ji
8. Dan Su

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, background): “We also briely summarize hallucinations in multi-modal settings such as visual-language tasks [1, 10] and speech-to-text tasks [119, 124].”