Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

By Yunsu Kim, P. Petrov, Pavel Petrushkov, Shahram Khadivi, Hermann Ney. 2019

94 citations recorded by the source corpus.

We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods greatly outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. We show that our improvements are valid also in zero-shot/zero-resource scenarios.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation (2020, methodology): “We compare with pivot-based translation and two recent approaches from Arivazhagan et al. (2019a) and Kim et al. (2019).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “While some researchers [19, 136] consider zero-shot to be synonymous with extremely LR case, others [91, 99] disagree.”
Findings of the 2021 Conference on Machine Translation (WMT21) (2021, unknown): “(Zoph et al., 2016; Dholakia and Sarkar, 2014; Kim et al., 2019).”
A Survey of Multilingual Neural Machine Translation (2019, methodology): “[72] combined S-P encoder with P-T decoder to create the S-T model.”