Effective Strategies in Zero-Shot Neural Machine Translation

By Thanh-Le Ha, Jan Niehues, A. Waibel. 2017

39 citations recorded by the source corpus.

In this paper, we proposed two strategies which can be applied to a multilingual neural machine translation system in order to better tackle zero-shot scenarios despite not having any parallel corpus. The experiments show that they are effective in terms of both performance and computing resources, especially in multilingual translation of unbalanced data in real zero-resourced condition when they alleviate the language bias problem.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tagged Back-Translation (2019, methodology): “We focus on back-translation (BT), which, despite its simplicity, has thus far been the most effective technique (Sennrich et al., 2017; Ha et al., 2017; Garc´ıa-Mart´ınez et al., 2017).”
A neural interlingua for multilingual machine translation (2018, neutral): “Nonetheless, in line with the results reported by (Miura et al., 2017; Ha et al., 2017) and others, we have also observed that direct neural zero-shot methods are not as effective as pivot translation at this time.”
The Missing Ingredient in Zero-Shot Neural Machine Translation (2019, background): “This phenomenon has been reported before (Ha et al., 2017b, 2016).”
Neural Machine Translation: A Review (2019, background): “On one side of the spectrum, the entire neural architecture (both encoder and decoder) can be shared, and source and target languages can be specified by annotating sentences [510] or words [511, 512] with language ID tags or embeddings.”
Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation (2018, background): “This approach yields state-of-the-art results even when large parallel data are available and has become common practice in NMT (Sennrich et al., 2017; Garcı́a-Martı́nez et al., 2017; Ha et al., 2017).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “An alternative is to add the language name as an input feature [67].”
A Survey of Multilingual Neural Machine Translation (2019, background): “Zero-shot NMT tends to translate into the wrong language at times and Ha et al. (2017) proposed to ﬁlter the output of the softmax so as to force the model to translate into the desired language.”
Improving Zero-shot Translation with Language-Independent Constraints (2019, methodology): “Subsequently, stronger constraints were bestowed upon the decoder to force the correct language to be generated through language features or vocabulary filtering during decoding (Ha et al., 2017).”
Consistency by Agreement in Zero-Shot Neural Machine Translation (2019, background): “…model and prepend special symbols to the source text to indicate the target language, which has later been extended to other text preprocessing approaches (Ha et al., 2017) as well as language-conditional parameter generation for encoders and decoders of a single model (Platanios et al., 2018).”
Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation (2019, methodology): “Recent approaches to zero-shot machine translation include pivot-based methods (Chen et al., 2017; Zheng et al., 2017; Cheng et al., 2016) and multi-lingual NMT methods (Firat et al., 2016a,b; Johnson et al., 2016; Ha et al., 2016, 2017).”