Generalized Data Augmentation for Low-Resource Translation

By Mengzhou Xia, X. Kong, Antonios Anastasopoulos, Graham Neubig. 2019

128 citations recorded by the source corpus.

Low-resource language pairs with a paucity of parallel data pose challenges for machine translation in terms of both adequacy and fluency. Data augmentation utilizing a large amount of monolingual data is regarded as an effective way to alleviate the problem. In this paper, we propose a general framework of data augmentation for low-resource machine translation not only using target-side monolingual data, but also by pivoting through a related high-resource language. Specifically, we experiment with a two-step pivoting method to convert high-resource data to the low-resource language, making best use of available resources to better approximate the true distribution of the low-resource language. First, we inject low-resource words into high-resource sentences through an induced bilingual dictionary. Second, we further edit the high-resource data injected with low-resource words using a modified unsupervised machine translation framework. Extensive experiments on four low-resource datasets show that under extreme low-resource settings, our data augmentation techniques improve translation quality by up to 1.5 to 8 BLEU points compared to supervised back-translation baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Machine Translation

Citations in the corpus

FFR v1.1: Fon-French Neural Machine Translation (2020, background): “Speciﬁcally, we are looking into leveraging monolingual data, encoding with subword units (Sennrich et al., 2015), exploring data augmentation for low-resource NMT (Fadaee et al., 2017; Xia et al., 2019), and training on a state-of-the-art Transformer model (Vaswani et al., 2017).”
How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers? (2020, background): “…have usually been observed where the augmentations are suited to the task: as with back-translation for machine translation (Edunov et al., 2018; Xia et al., 2019), or negative sampling for question answering and document retrieval (Zhang et al., 2017; Yang et al., 2019a; Xiong et al., 2020).”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Although high-resource language data can be leveraged for LRL with BT, success depends on factors such as relatedness of the high-resource language and LRL [95] or the availability of bilingual lexicons [198].”
DUB: Discrete Unit Back-translation for Speech Translation (2023, background): “…challenges faced by ST is data scarcity, which is similar to the low-resource scenarios encountered in MT. Intuitively, techniques developed for low-resource MT (Imamura et al., 2018; Xia et al., 2019; Chen et al., 2020; Liu et al., 2020 et al., 2020) should be utilized to improve ST performance.”
Adapting Multilingual Neural Machine Translation to Unseen Languages (2019, methodology): “More recently, a data augmentation strategy is proposed to further improve the LRL pairs [9].”
An Analysis of Simple Data Augmentation for Named Entity Recognition (2020, methodology): “Xia et al. (2019) convert data from a high-resource language to a low-resource language, using a bilingual dictionary and an unsupervised machine translation model in order to expand the machine translation training set for the low-resource language.”
Building Machine Translation Systems for the Next Thousand Languages (2022, background): “Dictionaries are relatively widely available and have already yielded promising results for individual language pairs (Xia et al., 2019; Duan et al., 2020; Karamanolakis et al., 2020; Reid et al., 2021).”
Syntax-Aware Data Augmentation for Neural Machine Translation (2020, methodology): “DATA augmentation (DA) is a helpful strategy in deep learning to enhance accuracy and has been widely used in computer vision (CV) [1], natural language processing (NLP) [2], [3], [4], [5], [6], [7], [8], [9] and other areas.”
Substructure Substitution: Structured Data Augmentation for NLP (2021, background): “…has been shown effective for various natural language processing (NLP) tasks, such as machine translation (Fadaee et al., 2017; Gao et al., 2019; Xia et al., 2019, inter alia ), text classiﬁcation (Wei and Zou, 2019; Quteineh et al., 2020), semantic role labeling (Fürstenau and Lap-ata, 2009)…”