Learning Hidden Unit Contribution for Adapting Neural Machine Translation Models
By David Vilar. 2018
38 citations recorded by the source corpus.
In this paper we explore the use of Learning Hidden Unit Contribution for the task of neural machine translation. The method was initially proposed in the context of speech recognition for adapting a general system to the specific acoustic characteristics of each speaker. Similar in spirit, in a machine translation framework we want to adapt a general system to a specific domain. We show that the proposed method achieves improvements of up to 2.6 BLEU points over a general system, and up to 6 BLEU points if the initial system has only been trained on out-of-domain data, a situation which may easily happen in practice. The good performance together with its short training time and small memory footprint make it a very attractive solution for domain adaptation.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Felix Stahlberg
- Bill Byrne
- Danielle Saunders
- Philipp Koehn
- Orhan Firat
- Ankur Bapna
- Kevin Duh
- Naveen Arivazhagan
In Natural Language Processing
- Felix Stahlberg
- Bill Byrne
- Orhan Firat
- Ankur Bapna
- Danielle Saunders
- Philipp Koehn
- Brian Thompson
- Naveen Arivazhagan
Citations in the corpus
- Simple, Scalable Adaptation for Neural Machine Translation (2019, methodology): “We first compare the adaptation performance of the light-weight residual adapters against full fine-tuning and LHUC (Vilar, 2018) on a large scale English-French domain adaptation task.”
- Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation (2019, methodology): “We do not assume that the domain of input sentences is known, thus we do not compare to methods such as LHUC (Vilar, 2018).”
- Domain Adaptive Inference for Neural Machine Translation (2019, background): “Vilar (2018) regularizes parameters with an importance network, while Thompson et al. (2018) freeze subsets of the model parameters before finetuning.”
- Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation (2018, background): “In a similar vein, other works (Swietojanski and Renals, 2014; Vilar, 2018) adapt a network to a new domain by learning additional weights that re-scale the hidden units.”
- Neural Machine Translation: A Review (2019, background): “by freezing sub-networks [486] or by only learning additional scaling factors for hidden units rather than full weights [487, 488].”