Lexically Constrained Neural Machine Translation with Levenshtein Transformer

By Raymond Hendy Susanto, Shamil Chollampatt, Liling Tan. 2020

87 citations recorded by the source corpus.

This paper proposes a simple and effective algorithm for incorporating lexical constraints in neural machine translation. Previous work either required re-training existing models with the lexical constraints or incorporating them during beam search decoding with significantly higher computational overheads. Leveraging the flexibility and speed of a recently proposed Levenshtein Transformer model (Gu et al., 2019), our method injects terminology constraints at inference time without any impact on decoding speed. Our method does not require any modification to the training procedure and can be easily applied at runtime with custom dictionaries. Experiments on English-German WMT datasets show that our approach improves an unconstrained baseline and previous approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pascale Fung
2. Andrea Madotto
3. Wenliang Dai
4. Nayeon Lee
5. Yan Xu
6. Etsuko Ishii
7. Ziwei Ji
8. Dan Su

In Computer Vision and Pattern Recognition

1. Pascale Fung
2. Wenliang Dai
3. Ziwei Ji
4. Dan Su
5. Weijia Xu
6. Marine Carpuat
7. Tiezheng Yu
8. Yejin Bang

In Natural Language Processing

1. Pascale Fung
2. Nayeon Lee
3. Andrea Madotto
4. Etsuko Ishii
5. Yejin Bang
6. Marine Carpuat
7. Weijia Xu
8. Yan Xu

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, methodology): “[32, 70, 140, 161, 166, 197, 198] use constrained decoding to incorporate specific terminology to machine translation, but the above methods can be repurposed to mitigate hallucinations.”
• CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020, neutral): “[MeanPooling-CopyNet]: the hand of a sink being washed up [ConstLeven]: a hand strikes a sink to wash from his soap.”
• Rule-based Morphological Inflection Improves Neural Terminology Translation (2021, methodology): “To enhance constraint usage in NAR models, we adopt the techniques by Susanto et al. (2020): we prohibit deletions on constraint tokens or insertions within the constraint segments.”
• “Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021, unknown): “Miao et al. (2019); Zhang et al. (2020b); Susanto et al. (2020) explore insertion-based non-autoregressive decoding algorithms.”
• Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation (2021, unknown): “In order to completely retain the skeleton in the generated text, we follow Susanto et al. (2020) to enforce hard constraints through forbidding the 2 We do argmax from Equation (9) instead of sampling.”
• EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2020, methodology): “Enforcing hard constraints as in Susanto et al. (2020) increases the term usage by +8–10% and improves BLEU by +0.3–0.6 over LevT using soft constraints.”