An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
By Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang. 2021
244 citations recorded by the source corpus.
NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting, summarizing the landscape of methods (including token-level augmentations, sentence-level augmentations, adversarial augmentations, and hidden-space augmentations) and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks. Based on the results, we draw several conclusions to help practitioners choose appropriate augmentations in different settings and discuss the current challenges and future directions for limited data learning in NLP.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sebastian Gehrmann
- Mohit Bansal
- Noah A. Smith
- Graham Neubig
- Yejin Choi
- Percy Liang
- Jianfeng Gao
- Diyi Yang
In Computer Vision and Pattern Recognition
- Sebastian Gehrmann
- Shafiq R. Joty
- Eduard H. Hovy
- Diyi Yang
- Sebastian Ruder
- Xuezhi Wang
- Thomas Scialom
- Chengwei Qin
Citations in the corpus
- Continual Sequence Generation with Adaptive Compositional Modules (2022, methodology): “Inspired by interpolation-based data augmentation (Chen et al., 2020, 2021) and neural architecture search (Liu et al., 2018), we utilize Hidden State Mixing for module selection.”
- Is GPT-3 a Good Data Annotator? (2022, methodology): “One approach to address the need for labeled data is through data augmentation (Feng et al., 2021; Meng et al., 2022; Chen et al., 2023), which involves generating new data by modifying existing data points using transformations based on prior knowledge about the problem’s structure (Yang et al.,…”
- NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation (2021, background): “Infrastruktur, kartu data, dan hasil evaluasi ketahanan dipublikasikan tersedia secara gratis di GitHub untuk kepentingan para peneliti yang mengerjakan pembuatan parafrase, analisis ketahanan, dan NLP sumber daya rendah.”