The NiuTrans Machine Translation Systems for WMT19

By Bei Li, Yinqiao Li, Chen Xu, Ye Lin, Jiqiang Liu, Hui Liu, Ziyang Wang, Yuhao Zhang, Nuo Xu, Zeyang Wang and 7 more. 2019

59 citations recorded by the source corpus.

This paper described NiuTrans neural machine translation systems for the WMT 2019 news translation tasks. We participated in 13 translation directions, including 11 supervised tasks, namely EN↔{ZH, DE, RU, KK, LT}, GU→EN and the unsupervised DE↔CS sub-track. Our systems were built on Deep Transformer and several back-translation methods. Iterative knowledge distillation and ensemble+reranking were also employed to obtain stronger models. Our unsupervised submissions were based on NMT enhanced by SMT. As a result, we achieved the highest BLEU scores in {KK↔EN, GU→EN} directions, ranking 2nd in {RU→EN, DE↔CS} and 3rd in {ZH→EN, LT→EN, EN→RU, EN↔DE} among all constrained submissions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, neutral): “En ↔ Zh: we report results from Li et al. (2019) which contains single model BLEU results on WMT2019.”
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information (2020, methodology): “Pre-trained language models such as BERT have been highly effective for NLP tasks (Peters et al., 2018; Devlin et al., 2019; Radford et al., 2019; Conneau and Lample, 2019; Liu et al., 2019; Yang et al., 2019).”
Findings of the 2018 Conference on Machine Translation (WMT18) (2018, methodology): “NIUTRANS further improves the translation performance by 2.28–3.83 BLEU points from four aspects of model variations (larger inner-hidden-size in FFN, using ReLU and attention dropout, Swish activation function, relative positional representation), diverse ensemble decoding (ensemble decoding with u”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, methodology): “…(Chatterjee et al., 2019) • metrics (Ma et al., 2019) • quality estimation (Fonseca et al., 2019) • parallel corpus filtering (Koehn et al., 2019) • robustness (Li et al., 2019b) In the news translation task (Section 2), participants were asked to translate a shared test set, optionally…”
GigaST: A 10, 000-hour Pseudo Speech Translation Corpus (2022, methodology): “We follow the data filtering and pre-processing methods described in [16, 17], and utilize iterative sequence-level knowledge distillation [18, 19] and back translation [20] techniques to improve the performance of MT.”
Learning Light-Weight Translation Models from Deep Transformer (2020, background): “…improvements by creating direct pass from the low-level encoder layers to the decoder (Bapna et al. 2018; Wang et al. 2019; Wei et al. 2020; Wu et al. 2019b; Li et al. 2019), and proper initialization strategies (Zhang, Titov, and Sennrich 2019; Xu et al. 2020; Liu et al. 2020; Huang et al. 2020).”