Multilingual Neural Machine Translation for Zero-Resource Languages

By Surafel Melaku Lakew, Mattia Antonino Di Gangi, Marcello Federico. 2019

14 citations recorded by the source corpus.

In recent years, Neural Machine Translation (NMT) has been shown to be more effective than phrase-based statistical methods, thus quickly becoming the state of the art in machine translation (MT). However, NMT systems are limited in translating low-resourced languages, due to the significant amount of parallel data that is required to learn useful mappings between languages. In this work, we show how the so-called multilingual NMT can help to tackle the challenges associated with low-resourced language translation. The underlying principle of multilingual NMT is to force the creation of hidden representations of words in a shared semantic space across multiple languages, thus enabling a positive parameter transfer across languages. Along this direction, we present multilingual translation experiments with three languages (English, Italian, Romanian) covering six translation directions, utilizing both recurrent neural networks and transformer (or self-attentive) neural networks. We then focus on the zero-shot translation problem, that is how to leverage multi-lingual data in order to learn translation directions that are not covered by the available training material. To this aim, we introduce our recently proposed iterative self-training method, which incrementally improves a multilingual NMT on a zero-shot direction by just relying on monolingual data. Our results on TED talks data show that multilingual NMT outperforms conventional bilingual NMT, that the transformer NMT outperforms recurrent NMT, and that zero-shot NMT outperforms conventional pivoting methods and even matches the performance of a fully-trained bilingual system.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, background): “Other work focused on exceptionally low resource scenarios, such as fully zero-shot translation (Lakew et al., 2019; Gu et al., 2019; Liu et al., 2020).”
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020, positive): “Although backtranslation has been successfully applied to zero-shot translation (Firat et al., 2016b; Gu et al., 2019; Lakew et al., 2019), whether it works in the massively multilingual set-up remained an open question and we investigate it in our work.”
A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation (2018, background): “…difference between ZST A and NMT, proving once again that zero-shot translation built on the “language-flag” of M-NMT is really effective (Johnson et al., 2016): in fact, at most a slight performance degradation is recorded as the number of pairs used in training decreases (Lakew et al., 2017).”
One-to-Many Multilingual End-to-End Speech Translation (2019, background): “A single model with shared parameters is particularly appealing in low-resource scenarios [33, 29] as it performs a sort of transfer learning across languages.”
Neural Machine Translation for Low-resource Languages: A Survey (2021, unknown): “All of these architectures are based on either the recurrent model with attention [91] or the Transformer-based model [182], and the latter has shown to be better [109, 111, 160] and is the base for most of the recent multi-NMT architectures.”