Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation

By Jiawei Wu, Xin Eric Wang, William Yang Wang. 2019

41 citations recorded by the source corpus.

The overreliance on large parallel corpora significantly limits the applicability of machine translation systems to the majority of language pairs. Back-translation has been dominantly used in previous approaches for unsupervised neural machine translation, where pseudo sentence pairs are generated to train the models with a reconstruction loss. However, the pseudo sentences are usually of low quality as translation errors accumulate during training. To avoid this fundamental issue, we propose an alternative but more effective approach, extract-edit, to extract and then edit real sentences from the target monolingual corpora. Furthermore, we introduce a comparative translation loss to evaluate the translated target sentences and thus train the unsupervised translation systems. Experiments show that the proposed approach consistently outperforms the previous state-of-the-art unsupervised machine translation systems across two benchmarks (English-French and English-German) and two low-resource language pairs (English-Romanian and English-Russian) by more than 2 (up to 3.63) BLEU points.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Multilingual Denoising Pre-training for Neural Machine Translation (2020, neutral): “Wu et al. (2019) solve the problem by mining sentences from Wikipedia and using them as weakly supervised translation pairs.”
Cross-lingual Retrieval for Iterative Self-Supervised Training (2020, background): “[16, 32, 19, 18, 8]. Our work differs from this line of work in that we do not use backtranslation but instead retrieve the target sentence directly from a monolingual corpus. More recently, [29] and [37] start incorporating explicit cross-lingual data into pretraining. However, since the quality of”
Neural Machine Translation: A Review (2019, background): “[509] is an alternative to back-translation for unsupervised NMT that edits a sentence in the monolingual corpus rather than synthesize it from scratch.”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “The accuracy of attribute marker extraction, for example, is constantly improving across literature (Sudhakar, Upadhyay, and Maheswaran 2019) and different ways to extract attribute markers can be easily fused (Wu et al. 2019).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, methodology): “The majority of the early unsupervised techniques focused on high-resource languages that have monolingual data in abundance [158, 164, 170, 196, 203].”
Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring (2019, unknown): “Recently, unsupervised MT (Artetxe et al., 2018b,a; Conneau et al., 2018; Lample et al., 2018b; Wu et al., 2019), which can immensely reduce the reliance on parallel corpora, has been gaining more and more interest.”
Retrieval-Enhanced Machine Learning (2022, background): “Wu et al. [65] used a retrieval unit to enable unsupervised training of machine translations, i.e., using two monolingual corpora in the source and target languages with no alignment.”