The University of Edinburgh’s Neural MT Systems for WMT17

By Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, Philip Williams. 2017

187 citations recorded by the source corpus.

This paper describes the University of Edinburgh's submissions to the WMT17 shared news translation and biomedical translation tasks. We participated in 12 translation directions for news, translating between English and Czech, German, Latvian, Russian, Turkish and Chinese. For the biomedical task we submitted systems for English to Czech, German, Polish and Romanian. Our systems are neural machine translation systems trained with Nematus, an attentional encoder-decoder. We follow our setup from last year and build BPE-based models with parallel and back-translated monolingual training data. Novelties this year include the use of deep architectures, layer normalization, and more compact models due to weight tying and improvements in BPE segmentations. We perform extensive ablative experiments, reporting on the effectivenes of layer normalization, deep architectures, and different ensembling techniques.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “, 2014) on a single source-target language pair (Wu et al., 2016; Sennrich et al., 2017).”
Tagged Back-Translation (2019, methodology): “We focus on back-translation (BT), which, despite its simplicity, has thus far been the most effective technique (Sennrich et al., 2017; Ha et al., 2017; Garc´ıa-Mart´ınez et al., 2017).”
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures (2018, neutral): “In addition, we also compare to the best result reported for DE→EN, achieved by uedin-wmt17 (Sennrich et al., 2017), which is an ensemble of 4 different models and reranked with right-to-left models.”
Findings of the 2019 Conference on Machine Translation (WMT19) (2019, background): “…and CommonCrawl (Junczys-Dowmunt, 2018a), continued training on newstest15-18 (like JHU’s submission to WMT18, Koehn et al., 2018), reranking with R2L models (like Sennrich et al., 2017 or JunczysDowmunt, 2018b) and fixing quotation marks to match the German style (as many other teams did).”
Iterative Back-Translation for Neural Machine Translation (2018, methodology): “The deep system uses matches the setup of Edinburgh’s WMT 2017 system (Sennrich et al., 2017).”
Marian: Fast Neural Machine Translation in C++ (2018, methodology): “Marian grew out of a C++ re-implementation of Nematus (Sennrich et al., 2017b), and still maintains binary-compatibility for common models.”
Evaluating the morphological competence of Machine Translation Systems (2017, methodology): “, 2017), UEDIN NMT (Sennrich et al., 2017a) on Nematus (Sennrich et al.”
Domain Adaptive Inference for Neural Machine Translation (2019, methodology): “, 2017), with all data except ParaCrawl oversampled by 2 (Sennrich et al., 2017).”
A Study of Reinforcement Learning for Neural Machine Translation (2018, result): “For the monolingual dataset on Zh-En translation task, similar to (Sennrich et al., 2017), the Chinese monolingual data comes from LDC Chinese Gi-gaword (4th edition) and the English monolingual data comes from News Crawl 2016 articles.”
Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias (2020, methodology): “For Chinese, we use the pre-trained models provided by Sennrich et al. (2017) (E-WMT).”
Online Distilling from Checkpoints for Neural Machine Translation (2019, methodology): “We use joint BPE segmentation (Sennrich et al., 2017) to process the whole training data.”
Emergent Translation in Multi-Agent Communication (2017, background): “Remarkable successes have been achieved in natural language processing (NLP) via the use of supervised learning approaches on large-scale datasets (Bahdanau et al., 2015; Wu et al., 2016; Gehring et al., 2017; Sennrich et al., 2017).”
Microsoft’s Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data (2018, methodology): “Modeled after the description from Sennrich et al. (2017), we reuse the example available at https://github.”
Sockeye: A Toolkit for Neural Machine Translation (2017, methodology): “NEMATUS For NEMATUS we also used the network settings described in the WMT 2017 submission Sennrich et al. [2017a].”
Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals (2020, methodology): “Checkpoint ensembles are usually worse than independent ensembles 33 , but allow to use more models in the ensemble thanks to shorter training time.”
On the Linguistic Representational Power of Neural Machine Translation Models (2019, methodology): “In NMT training, using subword units such as byte-pair encoding (Sennrich, Haddow, and Birch 2016) has become a de facto standard in training competition grade systems (Pinnis et al. 2017; Sennrich et al. 2017).”
Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation (2018, background): “, 2017), state-of-the-art systems still operate at the level of single sentences (Sennrich et al., 2017; Vaswani et al., 2017; Hassan et al., 2018).”