Training Neural Machine Translation to Apply Terminology Constraints

By Georgiana Dinu, Prashant Mathur, Marcello Federico, Yaser Al-Onaizan. 2019

191 citations recorded by the source corpus.

This paper proposes a novel method to inject custom terminology into neural machine translation at run time. Previous works have mainly proposed modifications to the decoding algorithm in order to constrain the output to include run-time-provided target terms. While being effective, these constrained decoding methods add, however, significant computational overhead to the inference step, and, as we show in this paper, can be brittle when tested in realistic conditions. In this paper we approach the problem by training a neural MT system to learn how to use custom terminology when provided with the input. Comparative experiments show that our method is not only more effective than a state-of-the-art implementation of constrained decoding, but is also as fast as constraint-free decoding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, methodology): “[32, 70, 140, 161, 166, 197, 198] use constrained decoding to incorporate specific terminology to machine translation, but the above methods can be repurposed to mitigate hallucinations.”
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020, background): “Two related scenarios with our task is lexically constrained decoding and word ordering (Zhang and Clark, 2015; Hasler et al., 2018; Dinu et al., 2019; Hokamp and Liu, 2017; Puduppully et al., 2017; Miao et al., 2019).”
Gradient-based Constrained Sampling from Language Models (2022, methodology): “Terminology Constrained Translation We follow the setup in Dinu et al. (2019) and use an off-the-shelf English to German translation model by MarianMT (Junczys-Dowmunt et al., 2018) to translate a subset of WMT17 en-de test set (Bojar et al., 2017).”
Rule-based Morphological Inflection Improves Neural Terminology Translation (2021, methodology): “synthetic parallel data augmented with constraint terms on the source side (Song et al., 2019; Dinu et al., 2019).”
Controlling Conditional Language Models without Catastrophic Forgetting (2021, background): “Some approaches to tackle it include constrained decoding (Chatterjee et al., 2017; Hasler et al., 2018), adding the desired terminology as part of the source context (Dinu et al., 2019), among others.”
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2020, methodology): “Lexical constraints or preferences have previously been incorporated by re-training NMT models with constraints as inputs (Song et al., 2019; Dinu et al., 2019) or with constrained beam search that drastically slows down decoding (Hokamp and Liu, 2017; Post and Vilar, 2018).”
NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics (2021, methodology): “To achieve this goal, much recent work proposed constrained decoding algorithms (Chatterjee et al., 2017; Hokamp and Liu, 2017; Hasler et al., 2018; Hu et al., 2019, inter alia ) or specialized training (Dinu et al., 2019).”
Lexically Constrained Neural Machine Translation with Levenshtein Transformer (2020, methodology): “Note that our baseline LevT model with Transformer blocks of 6 layers is superior to that of Dinu et al. (2019) who used a 2-layer conﬁguration.”
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2020, methodology): “We experiment with several recent pre-trained language models, including GPT-2 (Radford et al., 2019), UniLM (Dong et al., 2019), UniLM-v2 (Bao et al., 2020), BERT-Gen (Bao et al., 2020), BART (Lewis et al., 2020), and T5 (Raffel et al., 2019).”