Domain Adaptation for Statistical Machine Translation with Monolingual Resources

By Nicola Bertoldi, Marcello Federico. 2009

193 citations recorded by the source corpus.

Domain adaptation has recently gained interest in statistical machine translation to cope with the performance drop observed when testing conditions deviate from training conditions. The basic idea is that in-domain training data can be exploited to adapt all components of an already developed system. Previous work showed small performance gains by adapting from limited in-domain bilingual data. Here, we aim instead at significant performance gains by exploiting large but cheap monolingual in-domain data, either in the source or in the target language. We propose to synthesize a bilingual corpus by translating the monolingual adaptation data into the counterpart language. Investigations were conducted on a state-of-the-art phrase-based system trained on the Spanish--English part of the UN corpus, and adapted on the corresponding Europarl data. Translation, re-ordering, and language models were estimated after translating in-domain texts with the baseline. By optimizing the interpolation of these models on a development set the BLEU score was improved from 22.60% to 28.10% on a test set.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tie-Yan Liu
2. Tao Qin
3. Graham Neubig
4. Felix Stahlberg
5. Yang Liu
6. Kyunghyun Cho
7. Philipp Koehn
8. Jianfeng Gao

In Natural Language Processing

1. Graham Neubig
2. Tao Qin
3. Tie-Yan Liu
4. Felix Stahlberg
5. Philipp Koehn
6. Yang Liu
7. Ondrej Bojar
8. Kyunghyun Cho

Citations in the corpus

• Tagged Back-Translation (2019, methodology): “Back-translation was originally introduced for phrase-based systems (Bertoldi and Federico, 2009; Bojar and Tamchyna, 2011), but flourished in NMT after work by Sennrich et al.”
• Semi-Supervised Learning for Neural Machine Translation (2016, background): “ual corpora for conventional SMT has attracted intensive attention in recent years. Several authors have introduced transductive learning to make full use of monolingual corpora (Ueffing et al., 2007; Bertoldi and Federico, 2009). They use an existing translation model to translate unseen source text”
• Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization (2018, methodology): “Specifically, (Bertoldi and Federico 2009; Lambert et al. 2011) have back-translated target-side monolingual data into the sourceside sentence to produce synthetic parallel data for phrasebased SMT.”
• Multilingual Neural Machine Translation for Zero-Resource Languages (2019, methodology): “Mechanisms like back-translating the target side of a single language pair have been used for domain adaptation (Bertoldi and Federico 2009) and more recently by Sennrich, Haddow, and Birch (2015a) to improve an NMT baseline model.”
• The RWTH Aachen University English-Romanian Machine Translation System for WMT 2016 (2016, methodology): “To increase the amount of usable training data for the phrase-based and the neural machine translation systems we translated part of the monolingual training data back to English in a similar way as described by (Bertoldi and Federico, 2009) and (Sennrich et al.”
• Adaptation of machine translation for multilingual information retrieval in the medical domain (2014, background): “Bertoldi and Federico [31] exploit large amounts of in-domain monolingual data to create synthetic parallel training corpora.”
• Improving Translation Model by Monolingual Data (2011, methodology): “Bertoldi and Federico (2009) used monolingual data for adapting existing translation models to translation of data from different domains.”
• Explaining and Generalizing Back-Translation through Wake-Sleep (2018, methodology): “Back-translation as a technique for semi-supervised machine translations dates to the phrase-based era; see Bertoldi and Federico (2009).”
• Adapting Neural Machine Translation with Parallel Synthetic Data (2017, methodology): “Other works used target-side corpora to build the synthetic parallel corpus (Bertoldi and Federico, 2009; Lambert et al., 2011).”
• Investigations on Translation Model Adaptation Using Monolingual Data (2011, methodology): “This approach was reported to obtain interesting improvements in the translations quality (Schwenk and Senellart, 2009; Bertoldi and Federico, 2009).”
• Learning Hidden Unit Contribution for Adapting Neural Machine Translation Models (2018, unknown): “…methods can of course still be applied as long as they are model-independent, e.g. data selection methods as shown in (van der Wees et al., 2017) or self-training approaches like (Bertoldi and Federico, 2009), of which back-translation (Sennrich et al., 2015) can be considered a special case.”