Explaining and Generalizing Back-Translation through Wake-Sleep
By Ryan Cotterell, Julia Kreutzer. 2018
41 citations recorded by the source corpus.
Back-translation has become a commonly employed heuristic for semi-supervised neural machine translation. The technique is both straightforward to apply and has led to state-of-the-art results. In this work, we offer a principled interpretation of back-translation as approximate inference in a generative model of bitext and show how the standard implementation of back-translation corresponds to a single iteration of the wake-sleep algorithm in our proposed model. Moreover, this interpretation suggests a natural iterative generalization, which we demonstrate leads to further improvement of up to 1.6 BLEU.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Markus Freitag
- David Grangier
- Isaac Caswell
- Luke S. Zettlemoyer
- Ciprian Chelba
- Jie Zhou
- Hermann Ney
- Graham Neubig
Citations in the corpus
- Tagged Back-Translation (2019, methodology): “We further investigate the effects of TaggedBT by performing one round of iterative back-translation (Cotterell and Kreutzer, 2018; Vu Cong Duy Hoang and Cohn, 2018; Niu et al., 2018), and find another difference between the different varieties of BT: NoisedBT and TaggedBT allow the model to…”
- Text Repair Model for Neural Machine Translation (2019, positive): “Iterative Back-translation Iterative back-translation (Zhang et al., 2018; Cotterell and Kreutzer, 2018; Vu Cong Duy Hoang and Cohn, 2018) is a joint training algorithm to enhance the effect of monolingual source and target data by iteratively boosting the source-to-target and target-to-source…”
- Diversifying Dialogue Generation with Non-Conversational Text (2020, methodology): “It approximately quantifies the KL divergence between them two (Kim and Rush, 2016; Cotterell and Kreutzer, 2018).”
- Generalizing Back-Translation in Neural Machine Translation (2019, background): “Cotterell and Kreutzer (2018) frame back-translation as a variational process, with the space of source sentences as the latent space.”
- APE at Scale and Its Implications on MT Evaluation Biases (2019, methodology): “Iterative Back-translation Iterative back-translation (Zhang et al., 2018; Cotterell and Kreutzer, 2018; Hoang et al., 2018) is a joint training algorithm to enhance the effect of monolingual source and target data by iteratively boosting the source-to-target and target-to-source translation models.”