Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions

By Marcin Junczys-Dowmunt, Tomasz Dwojak, Hieu T. Hoang. 2016

202 citations recorded by the source corpus.

In this paper we provide the largest published comparison of translation quality for phrase-based SMT and neural machine translation across 30 translation directions. For ten directions we also include hierarchical phrase-based MT. Experiments are performed for the recently published United Nations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus. In the second part of the paper we investigate aspects of translation speed, introducing AmuNMT, our efficient neural machine translation decoder. We demonstrate that current neural machine translation could already be used for in-production systems when comparing words-persecond ratios.

Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

In Natural Language Processing

Citations in the corpus

A Convolutional Encoder Model for Neural Machine Translation (2016, methodology): “The optimized RNNsearch model and C++ decoder described by Junczys-Dowmunt et al. (2016) translates 265.”
Sparse Communication for Distributed Gradient Descent (2017, methodology): “AmuNMT (Junczys-Dowmunt et al., 2016) is used for decoding with a beam size of 12.”
A Challenge Set Approach to Evaluating Machine Translation (2017, methodology): “Since then, controlled comparisons have used BLEU to show that NMT out-performs strong PBMT systems on 30 translation directions from the United Nations Parallel Corpus (Junczys-Dowmunt et al., 2016a), and on the IWSLT English-Arabic tasks (Durrani et al., 2016).”
Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation (2018, background): “It dominated the recent WMT shared task (Bojar et al., 2017), and is used commercially (Wu et al., 2016; Crego et al., 2016; Junczys-Dowmunt et al., 2016).”
Visualizing and Understanding Neural Machine Translation (2017, methodology): “NMT proves to outperform conventional statistical machine translation (SMT) significantly across a variety of language pairs (Junczys-Dowmunt et al., 2016) and becomes the new de facto method in practical MT systems (Wu et al., 2016).”
RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition (2018, background): “• Marian (Junczys-Dowmunt et al., 2016) is implemented directly in C++ for performance reasons.”
Neural Machine Translation (2020, unknown): “Further Readings Kalchbrenner and Blunsom (2013) build a comprehensive machine translation model by first encoding the source sentence with a convolutional neural network, and then generate the target sentence by reversing the process.”
The University of Cambridge’s Machine Translation Systems for WMT18 (2018, methodology): “Furthermore, we also report gains from MBR-based combination with a phrase-based SMT system.”
Pinyin as Subword Unit for Chinese-Sourced Neural Machine Translation (2017, background): “NMT systems have outperformed the state-of-the-art SMT model on various language pairs in terms of translation quality [13, 2, 7, 22, 21, 5].”
Sockeye: A Toolkit for Neural Machine Translation (2017, methodology): “• MARIAN [Junczys-Dowmunt et al., 2016] – a C++ toolkit developed at Adam Mickiewicz University (AMU) in Poznań and at the University of Edinburgh.”
Improving Lexical Choice in Neural Machine Translation (2017, background): “, 2017) are appealing for their single-model, end-to-end training process, and have demonstrated competitive performance compared to earlier statistical approaches (Koehn et al., 2007; Junczys-Dowmunt et al., 2016).”
Syntax-aware Neural Machine Translation Using CCG (2017, background): “Sequence-to-sequence neural machine translation (NMT) models (Sutskever et al., 2014; Cho et al., 2014b; Bahdanau et al., 2015) are state-of-the-art on a multitude of language-pairs (Sennrich et al., 2016a; Junczys-Dowmunt et al., 2016).”
Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals (2020, methodology): “In line with these advances, the ﬁ eld of MT has shifted to the use of deep-learning neural-based methods 8 – 11 , which replaced previous approaches, such as rule-based systems 12 or statistical phrase-based methods 13,14 .”
CUNI Submissions in WMT18 (2018, methodology): “, 2018)2 and Marian (where the translation part was formerly known as AmuNMT) (Junczys-Dowmunt et al., 2016) with pretrained English-to-Czech Nematus models.”