Iterative Back-Translation for Neural Machine Translation

By Cong Duy Vu Hoang, Philipp Koehn, Gholamreza Haffari, Trevor Cohn. 2018

342 citations recorded by the source corpus.

We present iterative back-translation, a method for generating increasingly better synthetic parallel data from monolingual data to train neural machine translation systems. Our proposed method is very simple yet effective and highly applicable in practice. We demonstrate improvements in neural machine translation quality in both high and low resourced scenarios, including the best reported BLEU scores for the WMT 2017 German↔English tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Complete Multilingual Neural Machine Translation (2020, neutral): “The zero-resource approaches, combined with iterative-back translation (Hoang et al., 2018) are quite powerful but their inefﬁciency is worth noting.”
WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, methodology): “This method of back-translation to create additional parallel data has been shown to be effective in improving the performance of neural machine translation systems (Sennrich et al., 2016; Hoang et al., 2018; Edunov et al., 2018).”
PEER: A Collaborative Language Model (2022, methodology): “…synthetic data closely resembles other self-training and bootstrapping approaches used e.g. in word sense disambiguation (Yarowsky, 1995), machine translation (Sennrich et al., 2016; Hoang et al., 2018), sequence generation (He et al., 2020a), and few-shot learning (Schick and Schütze, 2021a,b).”
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout (2019, methodology): “Back-translation Back translation (Sennrich et al., 2016), a popular semi-supervised learning method, has been well studied in neural machine translation (Hoang et al., 2018; Wang et al., 2018b; Edunov et al., 2018; Prabhumoye et al., 2018).”
Generalizing Back-Translation in Neural Machine Translation (2019, background): “Several efforts (Hoang et al., 2018; Edunov et al., 2018; Imamura et al., 2018) conﬁrm that this is in fact beneﬁcial.”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “This is a popular approach in machine translation where target sentences are back-translated into source sentences (Bojar and Tamchyna, 2011; Hoang et al., 2018).”
Abstract Text Summarization: A Low Resource Challenge (2019, positive): “The iterative back-translation (e.g. training back-translation systems multiple times) were also found effective in machine translation (Hoang et al., 2018).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, methodology): “This procedure is continued iteratively, where the same set of sentences is back-translated several times until no improvement is observed in both translation directions [10, 82].”
CUNI Submission for Low-Resource Languages in WMT News 2019 (2019, methodology): “Hoang et al. (2018) showed that backtranslation can be iterated and with the second round of back-translation, we improve the performance of both models.”
Soft Contextual Data Augmentation for Neural Machine Translation (2019, methodology): “This is different from previous techniques, such as back translation, that rely on monolingual data (Sennrich et al., 2015a; Gulcehre et al., 2015; Cheng et al., 2016; He et al., 2016; Hoang et al., 2018).”
Diversifying Dialogue Generation with Non-Conversational Text (2020, methodology): “The backward and forward translation are iteratively performed to close the gap between P f and P b (Hoang et al., 2018; Cotterell and Kreutzer, 2018).”
A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation (2020, positive): “suggest an iterative data augmentation procedure that continuously improves the quality of the back-translation and final systems [36].”
Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation (2020, methodology): “Future work should explore techniques like iterative back-translation (Hoang et al., 2018) for further improvement and scaling to larger model capacities and more languages (Arivazhagan et al., 2019b; Huang et al., 2019)”
Neural Machine Translation: A Review (2019, background): “Iterative approaches that back-translate with systems that were by themselves trained with back-translation can yield improvements (Hoang et al., 2018; Niu et al., 2018; Zhang et al., 2018c) although they are not widely used due to their computational costs.”