Multilingual Neural Machine Translation With Soft Decoupled Encoding

By Xinyi Wang, Hieu Pham, Philip Arthur, Graham Neubig. 2019

66 citations recorded by the source corpus.

Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages. However, there are still significant challenges in efficiently learning word representations in the face of paucity of data. In this paper, we propose Soft Decoupled Encoding (SDE), a multilingual lexicon encoding framework specifically designed to share lexical-level information intelligently without requiring heuristic preprocessing such as pre-segmenting the data. SDE represents a word by its spelling through a character encoding, and its semantic meaning through a latent embedding space shared by all languages. Experiments on a standard dataset of four low-resource languages show consistent improvements over strong multilingual NMT baselines, with gains of up to 2 BLEU on one of the tested languages, achieving the new state-of-the-art on all four language pairs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Machine Translation

Citations in the corpus

Generalized Data Augmentation for Low-Resource Translation (2019, methodology): “We also compare the results of our augmentation methods with other state-of-the-art methods that either perform improvements to modeling to improve the ability to do parameter sharing (Wang et al., 2019), or train on many different target languages simultaneously (Aharoni et al., 2019).”
Balancing Training for Multilingual Neural Machine Translation (2020, methodology): “In order to do so, we turn to a recently proposed method of Differentiable Data Selection (Wang et al., 2019b, DDS), a general purpose machine learning method that allows for weighting of training data to improve performance on a separate set of held-out data.”
A Survey of Multilingual Neural Machine Translation (2019, background): “Wang et al. (2018) explored multiple methods for supporting target languages: (a) target language tag at beginning of the decoder, (b) target language dependent positional embeddings, and (c) divide hidden units of each decoder layer into shared and language-dependent ones.”
ChatGPT and Open-AI Models: A Preliminary Review (2023, background): “ChatGPT’s ability to generate coherent and contextually relevant responses in multiple languages has the potential to break down language barriers and promote cross-cultural communication [8].”
Adapting Multilingual Neural Machine Translation to Unseen Languages (2019, methodology): “Another approach optimizes embeddings through character n-grams (i.e., soft decoupled encoding, SDE) [8].”
The University of Maryland’s Kazakh-English Neural Machine Translation System at WMT19 (2019, methodology): “…conﬁgurations of sub-word segmentations using different variants of the standard Byte-Pair Encoding (BPE) framework (Sennrich et al., 2016), and compare them with the Soft Decoupled Encoding framework that exploits character n -gram representations of words instead of sub-words (Wang et al., 2019).”