FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN

By Ebrahim Ansari, Amittai Axelrod, Nguyen Bach, Ondrej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir Durrani, Marcello Federico, Christian Federmann, Jiatao Gu and 13 more. 2020

140 citations recorded by the source corpus.

The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation. A total of teams participated in at least one of the tracks. This paper introduces each track’s goal, data and evaluation metrics, and reports the results of the received submissions.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Juan Pino
2. Changhan Wang
3. Jiatao Gu
4. Yun Tang
5. Anne Wu
6. Xutai Ma
7. Ye Jia
8. Michelle Tadmor Ramanovich

In Evaluation & Benchmarks

1. Juan Pino
2. Changhan Wang
3. Jiatao Gu
4. Anne Wu
5. Xutai Ma
6. Marco Turchi
7. Matteo Negri
8. Yun Tang

In Machine Translation

1. Juan Pino
2. Changhan Wang
3. Yun Tang
4. Anne Wu
5. Xutai Ma
6. Ye Jia
7. Jiatao Gu
8. Dmytro Okhonko

Citations in the corpus

• Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq (2020, methodology): “Besides traditional offline models, we also provide simultaneous ST models: the lower section in Table 2 presents the online models with wait-k policy, which was the baseline system in the IWSLT 2020 shared task on simultaneous ST (Ansari et al., 2020).”
• The Multilingual TEDx Corpus for Speech Recognition and Translation (2021, background): “While cascade models currently represent the state-of-the-art [41], particularly for low-resource language pairs [42], end-to-end models are closing the performance gap for high-resource settings, as shown in the trends of recent IWSLT evaluation campaigns [41, 43].”
• Translatotron 2: Robust direct speech-to-speech translation (2021, background): “Very recently, direct speech-to-text translation (ST) is rapidly emerging, and has outperformed the cascade of ASR and MT (Weiss et al., 2017; Jia et al., 2019a; Di Gangi et al., 2019; McCarthy et al., 2020; Wang et al., 2021b; Ansari et al., 2020; Anastasopoulos et al., 2021), which makes the casca”
• SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation (2020, methodology): “In addition, an earlier version of SIMULEVAL was used in the context of the first simultaneous speech translation shared task at IWSLT (Ansari et al., 2020), where it is of paramount importance to have the same evaluation conditions for all submissions.”
• Translatotron 2: High-quality direct speech-to-speech translation with voice preservation (2021, background): “…emerging, and has outperformed the cascade of ASR and MT (Weiss et al., 2017; Jia et al., 2019a; Di Gangi et al., 2019; McCarthy et al., 2020; Ansari et al., 2020; Wang et al., 2021b; Anastasopoulos et al., 2021), which makes the Proceedings of the 39 th International Conference on Machine…”