Enhancement of Encoder and Attention Using Target Monolingual Corpora in Neural Machine Translation

By Kenji Imamura, Atsushi Fujita, Eiichiro Sumita. 2018

77 citations recorded by the source corpus.

A large-scale parallel corpus is required to train encoder-decoder neural machine translation. The method of using synthetic parallel texts, in which target monolingual corpora are automatically translated into source sentences, is effective in improving the decoder, but is unreliable for enhancing the encoder. In this paper, we propose a method that enhances the encoder and attention using target monolingual corpora by generating multiple source sentences via sampling. By using multiple source sentences, diversity close to that of humans is achieved. Our experimental results show that the translation quality is improved by increasing the number of synthetic source sentences for each given target sentence, and quality close to that using a manually created parallel corpus was achieved.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tagged Back-Translation (2019, background): “Like Imamura et al. (2018), they argue that BT through beam or greedy decoding leads to an overly regular domain on the source side, which poorly represents the diverse distribution of natural text.”
Data augmentation using back-translation for context-aware neural machine translation (2019, methodology): “The data augmentation in this study follows the existing back-translation strategies for NMT (Sennrich et al., 2016a; Imamura et al., 2018; Edunov et al., 2018) except that we assume a context-aware model for the forward-translation; the monolingual data for back-translation must be a set of…”
Neural Machine Translation: A Review (2019, background): “The variety of synthetic sentences in back-translation can also be increased by sampling multiple sentences from the reverse translation model (Imamura et al., 2018).”
Multi-Source Neural Machine Translation with Missing Data (2018, methodology): “proposed a method that generates multiple source sentences via sampling as an extension of the back-translation [18].”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Sampling: In sampling, multiple source sentences are generated per target sentence in an attempt to average out errors in synthetic sentences [65, 86].”
Generalizing Back-Translation in Neural Machine Translation (2019, methodology): “Several efforts (Hoang et al., 2018; Edunov et al., 2018; Imamura et al., 2018) conﬁrm that this is in fact beneﬁcial.”
Multi-Source Neural Machine Translation with Data Augmentation (2018, methodology): “Imamura et al. proposed a method that enhances the encoder and attention using target monolingual corpora by generating mutliple source sentences via sampling as an extension of the back-translation [15].”
A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation (2020, background): “On the contrary, decoding based on sampling or restricted sampling can produce diverse data by sampling from the model distribution [14,30].”
DUB: Discrete Unit Back-translation for Speech Translation (2023, background): “…challenges faced by ST is data scarcity, which is similar to the low-resource scenarios encountered in MT. Intuitively, techniques developed for low-resource MT (Imamura et al., 2018; Xia et al., 2019; Chen et al., 2020; Liu et al., 2020 et al., 2020) should be utilized to improve ST performance.”
slimIPL: Language-Model-Free Iterative Pseudo-Labeling (2020, methodology): “coding can be found in Section 5.3. Pseudo-labeling is also popular in computer vision [22, 60]. Variants exist, such as “soft labels” ^y = p (yjx) and variations on soft labeling [31, 61]. Sampling [62, 41] is also another way to generate pseudo-labels y^. 4 Language-Model-Free Iterative Pseudo-Lab”