Improving Domain Adaptation Translation with Domain Invariant and Specific Information
By Shuhao Gu, Yang Feng, Qun Liu. 2019
41 citations recorded by the source corpus.
In domain adaptation for neural machine translation, translation performance can benefit from separating features into domain-specific features and common features. In this paper, we propose a method to explicitly model the two kinds of information in the encoder-decoder framework so as to exploit out-of-domain data in in-domain training. In our method, we maintain a private encoder and a private decoder for each domain which are used to model domain-specific information. In the meantime, we introduce a common encoder and a common decoder shared by all the domains which can only have domain-independent information flow through. Besides, we add a discriminator to the shared encoder and employ adversarial training for the whole model to reinforce the performance of information separation and machine translation simultaneously. Experiment results show that our method can outperform competitive baselines greatly on multiple data sets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
Citations in the corpus
- Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation (2022, background): “Zeng et al. (2018); Gu et al. (2019) introduced a discriminator to preserve the domain-shared features.”
- Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation (2020, background): “Gu et al. (2019) adds a discriminator to help preserve the domain-shared features and fine tunes the whole model on the mixed training data.”
- Finding Sparse Structures for Domain Specific Neural Machine Translation (2021, background): “Fine-tuning is very common in domain adaptation, which first trains a base model on the general domain data and then fine-tunes it on each target domain (Luong and Manning 2015; Chen et al. 2017; Gu, Feng, and Liu 2019; Saunders et al. 2019).”
- Token-level Adaptive Training for Neural Machine Translation (2020, unknown): “In NMT, word frequency information is used for curriculum learning (Kocmi and Bojar, 2017; Zhang et al., 2018; Platanios et al., 2019) and domain adaptation data selection (Wang et al., 2017; Zhang and Xiong, 2018; Gu et al., 2019).”