Tagged Back-Translation

By Isaac Caswell, Ciprian Chelba, David Grangier. 2019

232 citations recorded by the source corpus.

Recent work in Neural Machine Translation (NMT) has shown significant quality gains from noised-beam decoding during back-translation, a method to generate synthetic parallel data. We show that the main role of such synthetic noise is not to diversify the source side, as previously suggested, but simply to indicate to the model that the given source is synthetic. We propose a simpler alternative to noising techniques, consisting of tagging back-translated source sentences with an extra token. Our results on WMT outperform noised back-translation in English-Romanian and match performance on English-German, redefining the state-of-the-art on the former.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “All other control codes are injected into the data without such special treatment (Moryossef et al., 2019; Caswell et al., 2019).”
Beyond English-Centric Multilingual Machine Translation (2020, methodology): “Following Caswell et al. (2019), we add a special encoder-side BT token to these translations to indicate to the model that they are synthetic.”
Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation (2019, methodology): “okens (subwords). Our models are trained with OpenNMTpy (Klein et al., 2017) on a single GTX 1080 Ti for 5 epochs. We experimented with ﬁne tuning on noisy data and mixed training with “domain” tags (Caswell et al., 2019; Kobus et al., 2016) indicatingwherethesentences aresourced from. We used diffe”
Translationese as a Language in “Multilingual” NMT (2019, methodology): “Finally, we perform several analyses of tagging the “languages” of translationese and demonstrate that tagged back-translation (Caswell et al., 2019), can be framed as a simplified version of our method, and thereby improved by targeted decoding.”
Using natural language prompts for machine translation (2022, unknown): “In particular, we use the results of Edunov et al. (2018) for English-French, the results of Wang et al. (2019) for English-German, and the results of Caswell et al. (2019) for English-Romanian.”
Facebook AI’s WAT19 Myanmar-English Translation Task Submission (2019, methodology): “We improve our baseline neural machine translation system with: tagging (Sennrich et al., 2016; Kobus et al., 2016; Caswell et al., 2019), ﬁne-tuning and ensembling.”
Facebook AI’s WMT20 News Translation Task Submission (2020, methodology): “Similarly, Caswell et al. (2019); Chen et al. (2019) show that adding specific tag to backtranslated and self-translated data can improve model performance.”
On Feature Normalization and Data Augmentation (2020, methodology): “Beyond vision tasks, back-translation [3, 8, 48, 66] and word dropout [25] are commonly used to augment text data.”
The Source-Target Domain Mismatch Problem in Machine Translation (2019, methodology): “tively by back-translation. Unfortunately, backtranslation is much less effective when dealing with STDM, as we will show in §6.1. Zheng et al. (2019) tackles this problem by adding tags to examples (Caswell et al., 2019) to let the model know whether the data originates from the source or target do”
DUB: Discrete Unit Back-translation for Speech Translation (2023, background): “Then, add special <BT> indicator at the begining of ˆ u ′ (Caswell et al., 2019).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “It has been shown that adding a tag to the back-translated data to distinguish them from the original data gives better results [22, 96, 131].”