An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation

By Chenhui Chu, Raj Dabre, Sadao Kurohashi. 2017

63 citations recorded by the source corpus.

In this paper, we propose a novel domain adaptation method named "mixed fine tuning" for neural machine translation (NMT). We combine two existing approaches namely fine tuning and multi domain NMT. We first train an NMT model on an out-of-domain parallel corpus, and then fine tune it on a parallel corpus which is a mix of the in-domain and out-of-domain corpora. All corpora are augmented with artificial tags to indicate specific domains. We empirically compare our proposed method against fine tuning and multi domain methods and discuss its benefits and shortcomings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Mohit Bansal
3. Felix Stahlberg
4. Philipp Koehn
5. Yonatan Belinkov
6. Kyunghyun Cho
7. Luke S. Zettlemoyer
8. Qun Liu

In Computer Vision and Pattern Recognition

1. Mohit Bansal
2. Graham Neubig
3. Felix Stahlberg
4. Yang Feng
5. Rico Sennrich
6. Quoc V. Le
7. Kyunghyun Cho
8. Philipp Koehn

In Natural Language Processing

1. Graham Neubig
2. Philipp Koehn
3. Felix Stahlberg
4. Mohit Bansal
5. Qun Liu
6. Kyunghyun Cho
7. Rico Sennrich
8. Ruslan Salakhutdinov

Citations in the corpus

• Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, methodology): “Following a practice in domain adaptation (Johnson et al., 2016; Chu et al., 2017), we append the”
• Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation (2020, background): “Chu et al. (2017) fine tunes the model with the mix of the general-domain data and over-sampled in-domain data.”
• Dynamic Data Selection for Neural Machine Translation (2017, methodology): “Other work combines fine-tuning with model ensembles (Freitag and Al-Onaizan, 2016) or with domain-specific tags in the training corpus (Chu et al., 2017).”
• Regularization techniques for fine-tuning in neural machine translation (2017, background): “In fact, Chu et al. (2017) successfully apply fine-tuning in combination with joint training.”
• Improving Domain Adaptation Translation with Domain Invariant and Specific Information (2019, methodology): “Chu et al. (2017) construct the training data set for the NMT model by combining out-of-domain data with the over-sampled in-domain data.”
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021, background): “Chu et al. (2017) fine-tunes the model with the mix of the general-domain data and over-sampled in-domain data.”
• Token-level Adaptive Training for Neural Machine Translation (2020, unknown): “• Sampler (Chu et al., 2017).”
• Sequence to Sequence Mixture Model for Diverse Machine Translation (2018, unknown): “Based on this factor, out-domain adaptation has been widely studied for MT, ranging from data selection (Li et al., 2010; Wang et al., 2017), tuning (Luong and Man-ning, 2015; Farajian et al., 2017) to domain tags (Chu et al., 2017).”
• Fine-Tuning for Neural Machine Translation with Limited Degradation across In- and Out-of-Domain Data (2017, methodology): “Another recent approach for NMT domain adaptation is the “mixed fine-tuning” by Chu et al. (2017).”