GenAug: Data Augmentation for Finetuning Text Generators
By Steven Y. Feng, Varun Prashant Gangal, Dongyeop Kang, Teruko Mitamura, Eduard H. Hovy. 2020
78 citations recorded by the source corpus.
In this paper, we investigate data augmentation for text generation, which we call GenAug. Text generation and language modeling are important tasks within natural language processing, and are especially challenging for low-data regimes. We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. We also examine the relationship between the amount of augmentation and the quality of the generated text. We utilize several metrics that evaluate important aspects of the generated text including its diversity and fluency. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods, and that the quality of generations improves to a peak at approximately three times the amount of original data.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Timo Schick
- Eduard H. Hovy
- Soroush Vosoughi
- Varun Prashant Gangal
- Tu Vu
- Jason Wei
- Teruko Mitamura
- Hinrich Schütze
In Natural Language Processing
- Varun Prashant Gangal
- Timo Schick
- Eduard H. Hovy
- Teruko Mitamura
- Soroush Vosoughi
- Jason Wei
- Steven Y. Feng
- Tu Vu
Citations in the corpus
- A Survey of Data Augmentation Approaches for NLP (2021, methodology): “, 2020) 3 × const+KWE Input 3 SYNTHETIC NOISE (Feng et al., 2020) × × tok Input 3 UEDIN-MS (DA part) (Grundkiewicz et al.”
- A Survey on Data Augmentation for Text Classification (2021, unknown): “[39] randomly delete, swap, and insert characters of texts (the prompt portion) that are used for fine-tuning text generators.”
- NAREOR: The Narrative Reordering Problem (2021, background): “Further, there is scope to explore use of narrative reordered inputs in downstream applications, such as data augmentation for tasks like paraphrase detection (Yin and Schütze, 2015) or text generation (Feng et al., 2020).”
- Leveraging QA Datasets to Improve Generative Data Augmentation (2022, background): “Therefore, for target domains that are different from the pretraining domain, such formulations may result in poor quality generation (Feng et al., 2020).”