Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination

By Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, Jianqiang Zhao. 2018

79 citations recorded by the source corpus.

With great practical value, the study of Multi-domain Neural Machine Translation (NMT) mainly focuses on using mixed-domain parallel sentences to construct a unified model that allows translation to switch between different domains. Intuitively, words in a sentence are related to its domain to varying degrees, so that they will exert disparate impacts on the multi-domain NMT modeling. Based on this intuition, in this paper, we devote to distinguishing and exploiting word-level domain contexts for multi-domain NMT. To this end, we jointly model NMT with monolingual attention-based domain classification tasks and improve NMT as follows: 1) Based on the sentence representations produced by a domain classifier and an adversarial domain classifier, we generate two gating vectors and use them to construct domain-specific and domain-shared annotations, for later translation predictions via different attention models; 2) We utilize the attention weights derived from target-side domain classifier to adjust the weights of target words in the training objective, enabling domain-related words to have greater impacts during model training. Experimental results on Chinese-English and English-French multi-domain translation tasks demonstrate the effectiveness of the proposed model. Source codes of this paper are available on Github https://github.com/DeepLearnXMU/WDCNMT.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation (2022, background): “Zeng et al. (2018); Gu et al. (2019) introduced a discriminator to preserve the domain-shared features.”
DEMix Layers: Disentangling Domains for Modular Language Modeling (2021, background): “Other works have explored multi-domain settings with smaller models and explicit domain labels, using supervision (e.g., Wright and Augenstein, 2020; Guo et al., 2018; Zeng et al., 2018) or dense training (e.”
Improving Domain Adaptation Translation with Domain Invariant and Specific Information (2019, background): “Zeng et al. (2018) extend this line of work by introducing a private encoder to learn some domain speciﬁc knowledge.”
Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021, background): “There are also some work (Zeng et al., 2018, 2019; Gu et al., 2019) that adds domain-specific modules to the model to preserve the domain-specific features.”
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing (2019, methodology): “In addition, Zeng et al. (2018) and Shen et al. (2017) propose a maximum weighted likelihood estimation method, where the weight is obtained by word-level domain aware masking to encourage the model to pay more attention to the domain-speciﬁc words.”