A Multilingual View of Unsupervised Machine Translation

By Xavier García, Pierre Foret, Thibault Sellam, Ankur P. Parikh. 2020

40 citations recorded by the source corpus.

We present a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups, focusing on unsupervised translation. In addition to studying the vanilla case where there is only monolingual data available, we propose a novel setup where one language in the (source, target) pair is not associated with any parallel data, but there may exist auxiliary parallel data that contains the other. This auxiliary data can naturally be utilized in our probabilistic framework via a novel cross-translation loss term. Empirically, we show that our approach results in higher BLEU scores over state-of-the-art unsupervised models on the WMT’14 English-French, WMT’16 English-German, and WMT’16 English-Romanian datasets in most directions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information (2020, methodology): “We highlight our contributions as follows: • We propose mRASP, an effective pre-training method that can be utilized to ﬁne-tune on any language pairs in NMT.”
Cross-lingual Retrieval for Iterative Self-Supervised Training (2020, background): “A major line of work that does not use any labeled parallel data typically works as follow: They ﬁrst train an initial language model using a noisy reconstruction objective, then ﬁnetune it using on-the-ﬂy backtranslation loss [16, 32, 19, 18, 8].”
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution (2021, methodology): “We train our models leveraging both monolingual and parallel datasets, following previous work (Siddhant et al., 2020; Garcia et al., 2020).”
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020, neutral): “We will also exploit novel strategies to break the upper bound of ROBT and obtain larger zero-shot improvements, such as generative modeling (Zhang et al., 2016; Su et al., 2018; García et al., 2020; Zheng et al., 2020).”
Pre-Trained Language Models for Text Generation: A Survey (2022, unknown): “[53] proposed using multi-task learning.”
Using natural language prompts for machine translation (2022, unknown): “These language pairs have been the focus of the recent literature on multilingual unsupervised machine translation (Wang et al., 2020; Li et al., 2020; Garcia et al., 2020, 2021; Chen et al., 2021) 4.”
Multilingual Translation from Denoising Pre-Training (2021, background): “…system between multiple languages (Firat et al., 2016; Johnson et al., 2017) has shown enormous improvement for translating low-resource languages (Gu et al., 2018), even enabling zero-shot translation (Lakew et al., 2018; Gu et al., 2019; Arivazhagan et al., 2019a; Garcia et al., 2020).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Recent back-translation based on multi-NMT models (Section 7) have shown much better results compared to the bilingual counterpart [60, 119, 164, 176, 186].”
Machine Translation into Low-resource Language Varieties (2021, background): “They include a combination of word-level and character-level MT (Vilar et al., 2007; Tiedemann, 2009; Nakov and Tiede-mann, 2012) Unsupervised translation models, which require only monolingual data, can address this limitation (Artetxe et al., 2018; Lample et al., 2018a; Garcia et al., 2020, 2021).”