Data Augmentation via Dependency Tree Morphing for Low-Resource Languages

By Gözde Gül Sahin, Mark Steedman. 2018

139 citations recorded by the source corpus.

Neural NLP systems achieve high scores in the presence of sizable training dataset. Lack of such datasets leads to poor system performances in the case low-resource languages. We present two simple text augmentation techniques using dependency trees, inspired from image processing. We “crop” sentences by removing dependency links, and we “rotate” sentences by moving the tree fragments around the root. We apply these techniques to augment the training sets of low-resource languages in Universal Dependencies project. We implement a character-level sequence tagging model and evaluate the augmented datasets on part-of-speech tagging task. We show that crop and rotate provides improvements over the models trained with non-augmented data for majority of the languages, especially for languages with rich case marking systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hai Zhao
2. Rui Wang
3. Dongdong Zhang
4. Sufeng Duan
5. Xiang Dai
6. Meng Jiang
7. Neil Shah
8. Heike Adel

In Computer Vision and Pattern Recognition

1. Rui Wang
2. Xiang Dai
3. Meng Jiang
4. Neil Shah
5. Heike Adel
6. Leonardo Neves
7. Yozen Liu
8. Hongyu Guo

In Natural Language Processing

1. Hai Zhao
2. Rui Wang
3. Dongdong Zhang
4. Sufeng Duan
5. Dietrich Klakow
6. Xiang Dai
7. Meng Jiang
8. Neil Shah

In Computational Linguistics & Semantics

1. Rui Wang
2. Hongyu Guo
3. Bernardo Magnini
4. Samuel Louvan
5. Dongdong Zhang
6. Hai Zhao
7. Sufeng Duan

Citations in the corpus

• A Survey of Data Augmentation Approaches for NLP (2021, neutral): “, 2020b) × × Input × DTREEMORPH (Şahin and Steedman, 2018) × × dep Input 3 Sub(2) (Shi et al.”
• LongForm: Effective Instruction Tuning with Reverse Instructions (2023, neutral): “Thus, our dataset features a diverse collection of instructions (including human-written, template-based and LLM-generated), each paired with a human-written output.”
• Data Augmentation Approaches in Natural Language Processing: A Survey (2021, unknown): “Rules [10], [7], [11] [100], [101] Machine Translation [42], [61], [10], [12], [60], [62], [64], [7], [19], [67], [102], [97], [103] [13], [59] [42], [58], [15], [104]”
• A Survey on Data Augmentation for Text Classification (2021, unknown): “Şahin and Steedman [83] are, for example, concerned with the augmentation of datasets from low resource languages for POS-tagging.”
• Syntax-Aware Data Augmentation for Neural Machine Translation (2020, methodology): “Steedman [47] proposed a dependency tree based method to generate training data by removing dependency links and moving the tree fragments around the root.”
• Edinburgh Research Explorer A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages (unknown): “Data augmentation is applicable in all scenarios, and has proven useful for low-resource NLP in general (Fadaee et al., 2017; Bergmanis et al., 2017; Sahin and Steedman, 2018).”
• Data Augmentation for Graph Neural Networks (2020, background): “…data without additional ground-truth labels, and have seen widespread adoption in fields such as computer vision (CV) (DeVries and Taylor 2017; Cubuk et al. 2019; Zhao et al. 2019; Ho et al. 2019), and natural language processing (NLP) (Fadaee, Bisazza, and Monz 2017; Şahin and Steedman 2019).”
• An Analysis of Simple Data Augmentation for Named Entity Recognition (2020, background): “…such as word replacement (Zhang et al., 2015; Wang and Yang, 2015; Cai et al., 2020), random deletion (Wei and Zou, 2019), or word position swap (S¸ahin and Steedman, 2018; Min et al., 2020); or create entirely artificial instances via generative models, such as variational auto encoders (Yoo et…”
• A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “Operations that (depending on the task) do not change the label include manipulation of parts of the dependency tree (Şahin and Steedman, 2018; Vania et al., 2019; Dehouck and Gómez-Rodríguez, 2020), sim-”
• Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification (2020, methodology): “alternative sentence structure through crop and rotate (Sahin and Steedman, 2018) operations based on a dependency parse structure.”
• Nonlinear Mixup: Out-Of-Manifold Data Augmentation for Text Classification (2020, methodology): “Some other NLP data augmentation methods are often devised for specific domains or require extra text resources (Sennrich, Had-dow, and Birch 2015; Fadaee, Bisazza, and Monz 2017; Sahin and Steedman 2018).”