Montreal Neural Machine Translation Systems for WMT’15

By Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio. 2015

159 citations recorded by the source corpus.

Neural machine translation (NMT) systems have recently achieved results comparable to the state of the art on a few translation tasks, including English→French and English→German. The main purpose of the Montreal Institute for Learning Algorithms (MILA) submission to WMT’15 is to evaluate this new approach on a greater variety of language pairs. Furthermore, the human evaluation campaign may help us and the research community to better understand the behaviour of our systems. We use the RNNsearch architecture, which adds an attention mechanism to the encoderdecoder. We also leverage some of the recent developments in NMT, including the use of large vocabularies, unknown word replacement and, to a limited degree, the inclusion of monolingual language models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Convolutional Sequence to Sequence Learning (2017, methodology): “For word-based models, we replace unknown words after generation by looking up the source word with the maximum attention score in a pre-computed dictionary (Jean et al., 2015).”
Neural Machine Translation of Rare Words with Subword Units (2015, methodology): “We generally follow settings by previous work (Bahdanau et al., 2015; Jean et al., 2015).”
Unsupervised Pretraining for Sequence to Sequence Learning (2016, neutral): “7 Supervised NMT (Jean et al., 2015) single 22.”
A Convolutional Encoder Model for Neural Machine Translation (2016, methodology): “Prior to scoring the generated translations against the respective references, we perform unknown word replacement based on attention scores (Jean et al., 2015).”
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism (2016, methodology): “…translation, neural machine translation, which is purely based on neural networks to perform full end-to-end translation, has become competitive with the existing phrase-based statistical machine translation in many language pairs (Jean et al., 2015; Gulcehre et al., 2015; Luong et al., 2015b).”
Zero-Resource Translation with Multi-Lingual Neural Machine Translation (2016, background): “In fact, if X1 and X2 are same and the two translation paths are simply two different models trained on the same language pair– direction, this is equivalent to constructing an ensemble, which was found to greatly improve translation quality (Sutskever et al., 2014; Jean et al., 2015)”
Character-based Neural Machine Translation (2016, background): “The proposed MT scheme provides improved results even when the source language is not morphologically rich.”
Controlling Politeness in Neural Machine Translation via Side Constraints (2016, methodology): “We train an attentional encoder-decoder NMT system using Groundhog 2 (Bahdanau et al., 2015; Jean et al., 2015a).”
Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization (2018, methodology): “(Jean et al. 2015) also reported experiments of reranking NMT outputs with a 5-gram language model.”
Paraphrasing Revisited with Neural Machine Translation (2017, methodology): “We trained a network for approximately 7 days on a single GPU, then the embedding layer was ﬁxed and training continued, as suggested in Jean et al. (2015a), for 12 hours.”
Neural CRF Model for Sentence Alignment in Text Simplification (2020, methodology): “If our model predicted <UNK>, we replaced it with the aligned source word (Jean et al., 2015).”
Correcting Length Bias in Neural Machine Translation (2018, methodology): “Length normalization divides the score by m (Koehn and Knowles, 2017; Jean et al., 2015; Boulanger-Lewandowski et al., 2013): Google’s NMT system (Wu et al., 2016) relies on a more complicated correction: Finally, some systems add a constant word reward (He et al., 2016): If γ = 0, this reduces to…”
Neural Machine Translation: A Review (2019, methodology): “Both softmax sampling and UNK replace have been used in one of the winning systems at the WMT’15 evaluation on English-German (Jean et al., 2015b).”