Automatically Learning Data Augmentation Policies for Dialogue Tasks

By Tong Niu, Mohit Bansal. 2019

40 citations recorded by the source corpus.

Automatic data augmentation (AutoAugment) (Cubuk et al., 2019) searches for optimal perturbation policies via a controller trained using performance rewards of a sampled policy on the target task, hence reducing data-level model bias. While being a powerful algorithm, their work has focused on computer vision tasks, where it is comparatively easy to apply imperceptible perturbations without changing an image’s semantic meaning. In our work, we adapt AutoAugment to automatically discover effective perturbation policies for natural language processing (NLP) tasks such as dialogue generation. We start with a pool of atomic operations that apply subtle semantic-preserving perturbations to the source inputs of a dialogue task (e.g., different POS-tag types of stopword dropout, grammatical errors, and paraphrasing). Next, we allow the controller to learn more complex augmentation policies by searching over the space of the various combinations of these atomic operations. Moreover, we also explore conditioning the controller on the source inputs of the target task, since certain strategies may not apply to inputs that do not contain that strategy’s required linguistic features. Empirically, we demonstrate that both our input-agnostic and input-aware controllers discover useful data augmentation policies, and achieve significant improvements over the previous state-of-the-art, including trained on manually-designed policies.

Subfields: Artificial Intelligence, Optimization & Training, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Kevin Gimpel
3. John Wieting
4. Luke S. Zettlemoyer
5. Diyi Yang
6. Jianfeng Gao
7. Colin Raffel
8. Karen Livescu

In Optimization & Training

1. Mohit Bansal
2. Derek Tam
3. Kevin Gimpel
4. Diyi Yang
5. Colin Raffel
6. Karen Livescu
7. Jiaao Chen
8. Freda Shi

Citations in the corpus

• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, background): “Similar attacks also help dialogue generation (Niu and Bansal, 2019) and text summarization (Cheng et al.”
• Substructure Substitution: Structured Data Augmentation for NLP (2021, background): “, 2020; Dehouck and Gómez-Rodrı́guez, 2020), semantic role labeling (Fürstenau and Lapata, 2009), and dialogue understanding (Hou et al., 2018; Niu and Bansal, 2019).”
• Data Augmentation Approaches in Natural Language Processing: A Survey (2021, unknown): “Non-Pretrained [26], [27], [80], [122], [123] [13], [79], [124], [125], [78] [80], [126]”