Substructure Substitution: Structured Data Augmentation for NLP
By Freda Shi, Karen Livescu, Kevin Gimpel. 2021
54 citations recorded by the source corpus.
We study a family of data augmentation methods, substructure substitution (SUB2), for natural language processing (NLP) tasks. SUB2 generates new examples by substituting substructures (e.g., subtrees or subsequences) with ones with the same label, which can be applied to many structured NLP tasks such as part-of-speech tagging and parsing. For more general tasks (e.g., text classification) which do not have explicitly annotated substructures, we present variations of SUB2 based on constituency parse trees, introducing structure-aware data augmentation methods to general NLP tasks. For most cases, training with the augmented dataset by SUB2 achieves better performance than training with the original training set. Further experiments show that SUB2 has more consistent performance than other investigated augmentation methods, across different tasks and sizes of the seed dataset.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sebastian Gehrmann
- Mohit Bansal
- Noah A. Smith
- Graham Neubig
- Eduard H. Hovy
- Yoav Goldberg
- Yejin Choi
- Sebastian Ruder
In Natural Language Processing
- Sebastian Gehrmann
- Thomas Scialom
- Wanxiang Che
- Sebastian Ruder
- Eduard H. Hovy
- Ondrej Dusek
- Yue Zhang
- Tongshuang Sherry Wu
Citations in the corpus
- A Survey of Data Augmentation Approaches for NLP (2021, neutral): “, 2020b) × × Input × DTREEMORPH (Şahin and Steedman, 2018) × × dep Input 3 Sub(2) (Shi et al., 2021) × × dep Input Substructural DAGA (Ding et al.”
- NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation (2021, methodology): “This transformation performs same-label subsequence substitution for the task of sequence tagging, which re-places a subsequence of the input tokens with another one that has the same sequence of tags (Shi et al., 2021).”
- A Survey on Data Augmentation for Text Classification (2021, unknown): “Certain works have challenged this perspective, by demonstrating that even existing patterns can be beneficial for performance [91].”
- Distribution Augmentation for Low-Resource Expressive Text-To-Speech (2022, unknown): “This is a popular data augmentation technique which not only works for low-resource text corpora but also shows gains for augmentation with massively pretrained models [10].”