Bi-Directional Neural Machine Translation with Synthetic Parallel Data

By Xing Niu, Michael J. Denkowski, Marine Carpuat. 2018

60 citations recorded by the source corpus.

Despite impressive progress in high-resource settings, Neural Machine Translation (NMT) still struggles in low-resource and out-of-domain scenarios, often failing to match the quality of phrase-based translation. We propose a novel technique that combines back-translation and multilingual NMT to improve performance in these difficult cases. Our technique trains a single model for both directions of a language pair, allowing us to back-translate source or target monolingual data without requiring an auxiliary model. We then continue training on the augmented parallel data, enabling a cycle of improvement for a single model that can incorporate any source, target, or parallel data to improve both translation directions. As a byproduct, these models can reduce training and deployment costs significantly compared to uni-directional models. Extensive experiments show that our technique outperforms standard back-translation in low-resource scenarios, improves quality on cross-domain tasks, and effectively reduces costs across the board.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tagged Back-Translation (2019, methodology): “…the effects of TaggedBT by performing one round of iterative back-translation (Cotterell and Kreutzer, 2018; Vu Cong Duy Hoang and Cohn, 2018; Niu et al., 2018), and ﬁnd another difference between the different varieties of BT: NoisedBT and TaggedBT allow the model to bootstrap improvements…”
Text Repair Model for Neural Machine Translation (2019, background): “A variety of flavors of iterative back-translation have been proposed, including Niu et al. (2018), who simultaneously perform iterative S2T and T2S backtranslation in a multilingual model, and Di He and Ma (2016); Yingce Xia and Liu (2017), who combine dual learning with phases of back- and…”
Bidirectional Learning for Domain Adaptation of Semantic Segmentation (2019, methodology): “The kind of techniques were first proposed to solve the neural machine translation problem, such as [10, 23], which train a language translation model for both directions of a language pair.”
APE at Scale and Its Implications on MT Evaluation Biases (2019, background): “A variety of ﬂavors of iterative back-translation have been proposed, including Niu et al. (2018), who simultaneously perform iterative S2T and T2S back-translation in a multilingual model, and He et al. (2016); Xia et al. (2017), who combine dual learning with phases of back-and…”
Neural Machine Translation: A Review (2019, background): “Iterative approaches that back-translate with systems that were by themselves trained with back-translation can yield improvements (Hoang et al., 2018; Niu et al., 2018; Zhang et al., 2018c) although they are not widely used due to their computational costs.”
A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation (2020, background): “Niu et al. use multilingual NMT, which trains two directions of a translation model in a single model to translate monolingual data that comes from source or target to generate synthetic data [37].”
Zero-Resource Neural Machine Translation with Monolingual Pivot Data (2019, background): “We also think that data selection methods on the back-translated data (Niu et al., 2018) could be helpful, since zero-shot multilingual NMT models often generate translations in the wrong target language (Arivazhagan et al., 2019).”