Unsupervised Pivot Translation for Distant Languages

By Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, Tie-Yan Liu. 2019

32 citations recorded by the source corpus.

Unsupervised neural machine translation (NMT) has attracted a lot of attention recently. While state-of-the-art methods for unsupervised translation usually perform well between similar languages (e.g., English-German translation), they perform poorly between distant languages, because unsupervised alignment does not work well for distant languages. In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation. We propose a learning to route (LTR) method to choose the translation path between the source and target languages. LTR is trained on language pairs whose best translation path is available and is applied on the unseen language pairs for path selection. Experiments on 20 languages and 294 distant language pairs demonstrate the advantages of the unsupervised pivot translation for distant languages, as well as the effectiveness of the proposed LTR for path selection. Specifically, in the best case, LTR achieves an improvement of 5.58 BLEU points over the conventional direct unsupervised method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Yingce Xia
5. Lijun Wu
6. Di He
7. Fei Tian
8. Renqian Luo

In Computer Vision and Pattern Recognition

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Yingce Xia
5. Lijun Wu
6. Fei Tian
7. Di He
8. Renqian Luo

In Natural Language Processing

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Yingce Xia
5. Lijun Wu
6. Di He
7. Fei Tian
8. Yiren Wang

Citations in the corpus

• MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, methodology): “Different from Artetxe et al. (2017); Lample et al. (2017; 2018); Leng et al. (2019), we just use back-translation to generate pseudo bilingual data for training, without using denoising auto-encoder 6 .”
• Microsoft Research Asia’s Systems for WMT19 (2019, background): “In addition to zero/low-resource setting (Leng et al., 2019), we also extend MASS to supervised setting where bilingual sentence pair ( x, y ) ∈ ( X , Y ) can be leveraged for pre-training.”
• Multilingual Neural Machine Translation with Language Clustering (2019, background): “…to cluster languages (Paul et al., 2009; Comrie, 1989; Bakker et al., 2009; Holman et al., 2008; Liu and Li, 2010; Chen and Gerdes, 2017; Wu et al., 2018; Leng et al., 2019), such as language family, language typology or other language characteristics from the URIEL database (Levin et al., 2017).”
• MTG: A Benchmark Suite for Multilingual Text Generation (2021, background): “Such direct parallel signal promotes knowledge transfer and cross-lingual generation between any language pairs (even distant pairs such as Spanish-Chinese) without involving an intermediate language such as English (Leng et al., 2019).”