Generalizing Back-Translation in Neural Machine Translation
By Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, Hermann Ney. 2019
54 citations recorded by the source corpus.
Back-translation — data augmentation by translating target monolingual data — is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German <-> English news translation task.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Varun Prashant Gangal
- Soroush Vosoughi
- Eduard H. Hovy
- Teruko Mitamura
- Jason Wei
- Steven Y. Feng
- Sarath Chandar
- Surangika Ranathunga
In Computer Vision and Pattern Recognition
- Jason Wei
- Sarath Chandar
- Surangika Ranathunga
- Eduard H. Hovy
- Ravi Shekhar
- Steven Y. Feng
- Mehreen Alam
- Teruko Mitamura
In Natural Language Processing
- Varun Prashant Gangal
- Eduard H. Hovy
- Teruko Mitamura
- Soroush Vosoughi
- Jason Wei
- Steven Y. Feng
- Sarath Chandar
- Surangika Ranathunga
Citations in the corpus
- A Survey of Data Augmentation Approaches for NLP (2021, background): “Graça et al. (2019) Investigate improvements to sampling-based approaches and the synthetic data generated by backtranslation.”
- Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Sampling: In sampling, multiple source sentences are generated per target sentence in an attempt to average out errors in synthetic sentences [65, 86].”