Zero-Resource Neural Machine Translation with Monolingual Pivot Data

By Anna Currey, Kenneth Heafield. 2019

36 citations recorded by the source corpus.

Zero-shot neural machine translation (NMT) is a framework that uses source-pivot and target-pivot parallel data to train a source-target NMT system. An extension to zero-shot NMT is zero-resource NMT, which generates pseudo-parallel corpora using a zero-shot system and further trains the zero-shot system on that data. In this paper, we expand on zero-resource NMT by incorporating monolingual data in the pivot language into training; since the pivot language is usually the highest-resource language of the three, we expect monolingual pivot-language data to be most abundant. We propose methods for generating pseudo-parallel corpora using pivot-language monolingual data and for leveraging the pseudo-parallel corpora to improve the zero-shot NMT system. We evaluate these methods for a high-resource language pair (German-Russian) using English as the pivot. We show that our proposed methods yield consistent improvements over strong zero-shot and zero-resource baselines and even catch up to pivot-based models in BLEU (while not requiring the two-pass inference that pivot models require).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Orhan Firat
2. Ankur Bapna
3. Naveen Arivazhagan
4. Raj Dabre
5. Chenhui Chu
6. Anoop Kunchukuttan
7. Graham Neubig
8. Yuan Cao

In Computer Vision and Pattern Recognition

1. Orhan Firat
2. Ankur Bapna
3. Naveen Arivazhagan
4. Raj Dabre
5. Yuan Cao
6. Chenhui Chu
7. Anoop Kunchukuttan
8. Yonghui Wu

In Machine Translation

1. Raj Dabre
2. Chenhui Chu
3. Ankur Bapna
4. Anoop Kunchukuttan
5. Orhan Firat
6. Yonghui Wu

Citations in the corpus

• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020, neutral): “, 2019) or pivotbased translation (Currey and Heafield, 2019).”
• A Survey of Multilingual Neural Machine Translation (2019, background): “Further, synthetic parallel corpus can be generated from monolingual pivot data, too [35].”
• Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation (2020, background): “, 2019a), it has not yet been shown how these massively multilingual models could be extended to unseen languages, beyond the pipelined approaches (Currey and Heafield, 2019; Lakew et al., 2019).”
• Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Pivot monolingual data is preferred because compared to source or target, pivot language would have much more monolingual data [35].”