Domain Control for Neural Machine Translation

By Catherine Kobus, Josep Maria Crego, Jean Senellart. 2016

197 citations recorded by the source corpus.

Machine translation systems are very sensitive to the domains they were trained on. Several domain adaptation techniques have already been deeply studied. We propose a new technique for neural machine translation (NMT) that we call domain control which is performed at runtime using a unique neural network covering multiple domains. The presented approach shows quality improvements when compared to dedicated domains translating on any of the covered domains and even on out-of-domain data. In addition, model parameters do not need to be re-estimated for each domain, making this effective to real use cases. Evaluation is carried out on English-to-French translation for two different testing scenarios. We first consider the case where an end-user performs translations on a known domain. Secondly, we consider the scenario where the domain is not known and predicted at the sentence level before translating. Results show consistent accuracy improvements for both conditions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “In this way, domain control codes receive special treatment (Kobus et al., 2016).”
Tagged Back-Translation (2019, methodology): “Most relevant to our work, Kobus et al. (2016) use tags to mark source sentence domain in a multi-domain setting.”
An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation (2017, background): “On the other hand, multi domain NMT (Kobus et al., 2016) involves training a single NMT model for multiple domains.”
A Survey of Domain Adaptation for Neural Machine Translation (2018, methodology): “For NMT, Kobus et al. (2016) propose an NMT domain control method, by appending either domain tags or features to the word embedding layer of NMT.”
Improving Robustness of Machine Translation with Synthetic Noise (2019, methodology): “Kobus et al. (2016) introduce a method of domain tagging to assist neural models in differentiating domains.”
Neural Machine Translation (2020, unknown): “Such tokens may represent the domain of the input sentence (Kobus et al., 2016), or the required level of politeness of the output sentence (Sennrich et al.”
Neural Machine Translation Training in a Multi-Domain Scenario (2017, methodology): “[9] proposed a domain aware concatenated system by introducing domain tags for every domain.”
Neural Machine Translation: A Review (2019, background): “sentences, possibly with domain-tags to help learning (Britz et al., 2017; Kobus et al., 2017; Tars & Fishel, 2018).”
Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation (2019, methodology): “We experimented with ﬁne tuning on noisy data and mixed training with “domain” tags (Caswell et al., 2019; Kobus et al., 2016) indicating where the sentences are sourced from.”
Translationese as a Language in “Multilingual” NMT (2019, background): “…models (Johnson et al., 2016), formality level in English→Japanese (Yamagishi et al., 2016), politeness in English→German (Sennrich et al., 2016a), gender from a gender-neutral language (Kuczmarski and Johnson, 2018), as well as to produce domain-targeted translation (Kobus et al., 2016).”
An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation (2017, methodology): “On the other hand, multi domainNMT (Kobus et al., 2016) involves training a single NMT model for multiple domains.”
Domain Robustness in Neural Machine Translation (2019, background): “…small amounts of parallel or monolingual in-domain data has proven very effective for NMT (e.g. Luong and Manning, 2015; Sennrich et al., 2016a; Kobus et al., 2017), the target domain(s) may be unknown when a system is built, and there are language pairs for which training data is only…”
Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021, unknown): “Kobus et al. (2017) adds domain-specific tags to each sentence.”
Improving Domain Adaptation Translation with Domain Invariant and Specific Information (2019, methodology): “In these two experiments, we only compared our method with the baseline model and the competitive contrast methods ’Sam-pler’ (Kobus et al., 2017) and ’Fine Tune’ (Lu-ong and Manning, 2015).”
Polite Dialogue Generation Without Parallel Data (2018, background): “16 These levels were deﬁned by Landis and Koch (1977)”
Facebook AI’s WAT19 Myanmar-English Translation Task Submission (2019, methodology): “We improve our baseline neural machine translation system with: tagging (Sennrich et al., 2016; Kobus et al., 2016; Caswell et al., 2019), ﬁne-tuning and ensembling.”
Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination (2018, background): “To tackle this problem, recently, researchers have carried out many constructive and in-depth studies (Kobus et al., 2016; Zhang et al., 2016; Pryzant et al., 2017; Farajian et al., 2017).”