Achieving Human Parity on Automatic Chinese to English News Translation

By Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William D. Lewis, Mu Li and 14 more. 2018

645 citations recorded by the source corpus.

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether such systems can approach or achieve parity with human translations. In this paper, we first address the problem of how to define and accurately measure human parity in translation. We then describe Microsoft's machine translation system and measure the quality of its translations on the widely used WMT 2017 news translation task from Chinese to English. We find that our latest neural machine translation system has reached a new state-of-the-art, and that the translation quality is at human parity when compared to professional human translations. We also find that it significantly exceeds the quality of crowd-sourced non-professional translations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, neutral): “…Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017; Tan et al., 2019; Artetxe et al., 2017; Lample et al., 2017; 2018; He et al., 2018; Hassan et al., 2018; Song et al., 2018; Shen et al., 2018), text summarization (Ayana et al., 2016; Suzuki & Nagata, 2017; Gehring et al., 2017),…”
Massively Multilingual Neural Machine Translation (2019, background): “Neural machine translation (NMT) (Kalchbren-ner and Blunsom, 2013; Bahdanau et al., 2014; Sutskever et al., 2014) is the current state-of-the-art approach for machine translation in both academia (Bojar et al., 2016, 2017, 2018) and industry (Wu et al., 2016; Hassan et al., 2018).”
Learning and Evaluating General Linguistic Intelligence (2019, neutral): “For example, machine performance on both Chinese–English machine translation and document question answering on the Stanford question answering dataset (SQuAD; Rajpurkar et al., 2016) has been claimed to have surpassed human levels (Hassan et al., 2018; Devlin et al., 2018).”
A Simple Method for Commonsense Reasoning (2018, background): “Although deep neural networks have achieved remarkable successes (e.g., [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]), their dependence on supervised learning has been challenged as a signiﬁcant weakness.”
Simple, Scalable Adaptation for Neural Machine Translation (2019, background): “While NMT performance on sentence level translation for high resource languages seems to be dramatically improved (Wu et al., 2016; Hassan et al., 2018), performance on out-of-domain data or low resource languages, still remains pretty poor (Duh et al.”
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “…2014; Bahdanau et al., 2014) have been widely adopted as the state-of-the-art approach for machine translation, both in the research community (Bojar et al., 2016a, 2017, 2018b) and for large-scale production systems (Wu et al., 2016; Zhou et al., 2016; Crego et al., 2016; Hassan et al., 2018).”
Microsoft Research Asia’s Systems for WMT19 (2019, methodology): “We follow previous practice (Hassan et al., 2018) and apply Byte-Pair Encoding (BPE) (Sennrich et al.”
Pay Less Attention with Lightweight and Dynamic Convolutions (2019, methodology): “For WMT English to Chinese (Zh-En), we pre-process the WMT’17 training data following Hassan et al. (2018) resulting in 20M sentence pairs.”
Phrase-Based & Neural Unsupervised Machine Translation (2018, neutral): “While recent advances have reported near human-level performance on several language pairs using neural approaches (Wu et al., 2016; Hassan et al., 2018), other studies have highlighted several open challenges (Koehn and Knowles, 2017; Isabelle et al., 2017; Sennrich, 2017).”
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (2018, methodology): “…encoder-decoder to train sentence embeddings on existing parallel data, which are then directly applied to retrieve and ﬁlter new parallel sentences using nearest neighbor retrieval over cosine similarity with a hard threshold (Espa˜na-Bonet et al., 2017; Hassan et al., 2018; Schwenk, 2018).”
Multilingual Neural Machine Translation with Knowledge Distillation (2019, background): “, 2017) and human parity achievements (Hassan et al., 2018).”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, background): “This motivation remains also this year but one more reason for people providing test suites was to examine the human parity question from additional viewpoints beyond what Bojar et al. (2018) discuss for English→Czech and Hassan et al. (2018) for Chinese→English.”