Findings of the 2021 Conference on Machine Translation (WMT21)

By Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondrej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta Ruiz Costa-jussà, Cristina España-Bonet, Angela Fan, Christian Federmann and 26 more. 2021

244 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yi Tay
2. Mohit Bansal
3. Sebastian Gehrmann
4. Quoc V. Le
5. Graham Neubig
6. Yoshua Bengio
7. Zhe Gan
8. Percy Liang

Citations in the corpus

• PaLM 2 Technical Report (2023, methodology): “WMT21 Experimental Setup We use the recent WMT 2021 sets (Akhbardeh et al., 2021) to guard against train/test data leakage, and to facilitate comparison with the state of the art.”
• Is ChatGPT A Good Translator? A Preliminary Study (2023, background): “…languages to evaluate the capability of ChatGPT in multilingual translation, including German (De), English (En), Romanian (Ro), and Chinese (Zh), which are commonly adopted in both research (Wang et al., 2022; Jiao et al., 2021, 2022b) and competitions (Bojar et al., 2016; Farhad et al., 2021).”
• DUB: Discrete Unit Back-translation for Speech Translation (2023, unknown): “…and Fed-erico, 2009; Bojar and Tamchyna, 2011) and neural machine translation models (Sennrich et al., 2016; Edunov et al., 2018; Hoang et al., 2018), and is frequently used to improve translation performance in WMT competitions (Farhad et al., 2021; Wen-zek et al., 2021; Adelani et al., 2022).”
• GigaST: A 10, 000-hour Pseudo Speech Translation Corpus (2022, methodology): “Table 1 illustrates the automated metrics evaluation (in BLEU) of our MT models for En-Zh and En-De directions on newstest2021 set, where Online-B/W and Online-W/A are top2 online systems individually for En-Zh and En-De reported by [24].”
• Dialect-robust Evaluation of Generated Text (2022, background): “The subcommunity that has attended most to dialect is the VarDial series of workshops, which has featured shared tasks such as dialect classification (Zampieri et al., 2014), translation between dialects (Akhbardeh et al., 2021), and transfer of NLP systems across dialects (Zampieri et al., 2017).”
• Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine (2023, background): “…languages to evaluate the capability of ChatGPT in multilingual translation, including German (De), English (En), Romanian (Ro), and Chinese (Zh), which are commonly adopted in both research (Wang et al., 2022a; Jiao et al., 2021, 2022b) and competitions (Bojar et al., 2016; Farhad et al., 2021).”
• How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation (2023, methodology): “We use source-based sentence-level contrastive Direct Assessment + Scalar Quality Metric (contrastive DA+SQM; Akhbardeh et al. 2021, Kocmi et al. 2022a) to perform human evaluation of the WMT-Best systems from Table 1 and GPT with 5 shots QR as shown in Table 3.”
• Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain (2021, methodology): “The WMT evaluation campaign (Akhbardeh et al., 2021) ran a human direct assessment (DA) evaluation for the primary submissions in the news domain for all language pairs.”
• Toxicity in Multilingual Machine Translation at Scale (2022, background): “Human measures use annotators to rank translation outputs, e.g., Licht et al. (2022); Akhbardeh et al. (2021).”