Textual Data Augmentation for Efficient Active Learning on Tiny Datasets

By Husam Quteineh, Spyridon Samothrakis, Richard F. E. Sutcliffe. 2020

50 citations recorded by the source corpus.

In this paper we propose a novel data augmentation approach where guided outputs of a language generation model, e.g. GPT-2, when labeled, can improve the performance of text classifiers through an active learning process. We transform the data generation task into an optimization problem which maximizes the usefulness of the generated output, using Monte Carlo Tree Search (MCTS) as the optimization strategy and incorporating entropy as one of the optimization criteria. We test our approach against a Non-Guided Data Generation (NGDG) process that does not optimize for a reward function. Starting with a small set of data, our results show an increased performance with MCTS of 26% on the TREC-6 Questions dataset, and 10% on the Stanford Sentiment Treebank SST-2 dataset. Compared with NGDG, we are able to achieve increases of 3% and 5% on TREC-6 and SST-2.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kevin Gimpel
2. Eduard H. Hovy
3. Mohit Bansal
4. Karen Livescu
5. Graham Neubig
6. John Wieting
7. Varun Prashant Gangal
8. Wanxiang Che

In Computer Vision and Pattern Recognition

1. Kevin Gimpel
2. Karen Livescu
3. Eduard H. Hovy
4. Teruko Mitamura
5. Steven Y. Feng
6. Varun Prashant Gangal
7. Sarath Chandar
8. Dacheng Tao

In Natural Language Processing

1. Kevin Gimpel
2. Eduard H. Hovy
3. Karen Livescu
4. Varun Prashant Gangal
5. Teruko Mitamura
6. Wanxiang Che
7. Mohit Bansal
8. John Wieting

In Representation & Self-Supervised Learning

1. Kevin Gimpel
2. Karen Livescu
3. Eduard H. Hovy
4. Varun Prashant Gangal
5. Steven Y. Feng
6. Teruko Mitamura
7. Jason Wei
8. Sarath Chandar

Citations in the corpus

• A Survey of Data Augmentation Approaches for NLP (2021, methodology): “Quteineh et al. (2020) use a similar label-conditioned GPT-2 generation method, and demonstrate its effectiveness as a DA method in an active learning setup.”
• Data Augmentation Approaches in Natural Language Processing: A Survey (2021, background): “Quteineh et al. [34] use label-conditioned GPT-2 to generate augmented data.”
• Substructure Substitution: Structured Data Augmentation for NLP (2021, methodology): “, 2019, inter alia), text classification (Wei and Zou, 2019; Quteineh et al., 2020), syntactic and semantic parsing (Jia and Liang, 2016; Shi et al.”