Findings of the WMT 2020 Shared Task on Automatic Post-Editing
By Rajen Chatterjee, Markus Freitag, Matteo Negri, Marco Turchi. 2020
37 citations recorded by the source corpus.
We present the results of the 6th round of the WMT task on MT Automatic Post-Editing. The task consists in automatically correcting the output of a “black-box” machine translation system by learning from existing human corrections of different sentences. This year, the challenge consisted of fixing the errors present in English Wikipedia pages translated into German and Chinese by state-ofthe-art, not domain-adapted neural MT (NMT) systems unknown to participants. Six teams participated in the English-German task, submitting a total of 11 runs. Two teams participated in the English-Chinese task submitting 2 runs each. Due to i) the different source/domain of data compared to the past (Wikipedia vs Information Technology), ii) the different quality of the initial translations to be corrected and iii) the introduction of a new language pair (English-Chinese), this year’s results are not directly comparable with last year’s round. However, on both language directions, participants’ submissions show considerable improvements over the baseline results. On English-German, the top ranked system improves over the baseline by -11.35 TER and +16.68 BLEU points, while on EnglishChinese the improvements are respectively up to -12.13 TER and +14.57 BLEU points. Overall, coherent gains are also highlighted by the outcomes of human evaluation, which confirms the effectiveness of APE to improve MT quality, especially in the new generic domain selected for this year’s round.
Subfields: Artificial Intelligence, Evaluation & Benchmarks, Machine Translation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Philipp Koehn
- Ondrej Bojar
- Graham Neubig
- Felix Stahlberg
- Markus Freitag
- Luke S. Zettlemoyer
- Barry Haddow
- Yonatan Belinkov
In Evaluation & Benchmarks
- Ondrej Bojar
- Philipp Koehn
- Markus Freitag
- Yvette Graham
- Barry Haddow
- Christian Federmann
- Marco Turchi
- Matteo Negri
In Machine Translation
- Philipp Koehn
- Barry Haddow
- Matteo Negri
- Marco Turchi
- Markus Freitag
- Ondrej Bojar
- Rajen Chatterjee
- Christian Federmann
Citations in the corpus
- Findings of the 2021 Conference on Machine Translation (WMT21) (2021, background): “Similar to last year’s task (Chatterjee et al., 2020), all 4 submissions significantly improved the original MT output.”