Multi-way, multilingual neural machine translation

By Orhan Firat, Kyunghyun Cho, Baskaran Sankaran, Fatos T. Yarman-Vural, Yoshua Bengio. 2017

100 citations recorded by the source corpus.

The first attention-based neural-MT for multi-way, multilingual translation is proposed.Multi-way multilingual model is tested on more than 8 languages (En, Fr, Cz, De, Ru, Fi, Tr and Uz).It achieves the translation quality comparable to single-pair NMTs with less parameters.Single attention mechanism supports to align between multiple pairs and directions.Outperforms conventional SMT system on low-resource translation tasks. We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT15 simultaneously and observe clear performance improvements over models trained on only one language pair. We empirically evaluate the proposed model on low-resource language translation tasks. In particular, we observe that the proposed multilingual model outperforms strong conventional statistical machine translation systems on Turkish-English and Uzbek-English by incorporating the resources of other language pairs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, methodology): “For the base-line experiment, we start by following common conventions in literature on multilingual NMT (Firat et al., 2017; Lee et al., 2017; Johnson et al., 2017).”
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation (2016, background): “Other approaches to multilingual translation require complex update scheduling mechanisms to prevent this effect (Firat et al., 2016b).”
Assessing gender bias in machine translation: a case study with Google Translate (2018, methodology): “s, making a re-entrance in the 1970s before the 1980s surge in statistical methods for machine translation [25, 26]. Statistical and example-based machine translation have been on the rise ever since [2, 8, 14], with highly successful applications such as Google Translate (recently ported to a neura”
Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors (2017, background): “[11] have additionally demonstrated NMT’s ability to translate between multiple language pairs with a neural translation model trained with a single attention mechanism.”
Why Catalan-Spanish Neural Machine Translation? Analysis, comparison and combination with standard Rule and Phrase-based technologies (2017, background): “…ones mentioned in previous: Byte Pair Encoding (BPE) (Sennrich et al., 2016), character-based (Costa-juss`a and Fonollosa, 2016), coverage (Tu et al., 2016), language model (Gulcehre et al., 2017), multilingual (Firat et al., 2017) and other strategies (Wu et al., 2016) affect this language pair.”
A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation (2020, background): “Using high-resource language to help improve the performance of low-resource neural machine translation is an intuitive approach; for example, transfer learning [5], model-agnostic meta-learning algorithm [6], triangular architecture [7], and multi-way, multilingual NMT frameworks [8].”
Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters (2020, background): “Architectures in this category may vary between sharing some layers (Firat et al., 2017; Lu et al., 2018) or no sharing at all (Escolano et al.”