Analyzing Dynamic Adversarial Training Data in the Limit

By Eric Wallace, Adina Williams, Robin Jia, Douwe Kiela. 2021

34 citations recorded by the source corpus.

To create models that are robust across a wide range of test inputs, training datasets should include diverse examples that span numerous phenomena. Dynamic adversarial data collection (DADC), where annotators craft examples that challenge continually improving models, holds promise as an approach for generating such diverse training sets. Prior work has shown that running DADC over 1-3 rounds can help models fix some error types, but it does not necessarily lead to better generalization beyond adversarial test data. We argue that running DADC over many rounds maximizes its training-time benefits, as the different rounds can together cover many of the task-relevant phenomena. We present the first study of longer-term DADC, where we collect 20 rounds of NLI examples for a small set of premise paragraphs, with both adversarial and non-adversarial approaches. Models trained on DADC examples make 26% fewer errors on our expert-curated test set compared to models trained on non-adversarial data. Our analysis shows that DADC yields examples that are more difficult, more lexically and syntactically diverse, and contain fewer annotation artifacts compared to non-adversarial examples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Some of these efforts augment humans (through guidelines, templates, programmatic generation of attacks, and various combinations thereof) to devise test cases that cause systems to fail [45, 46, 29, 21, 30, 55, 6, 23].”
Red Teaming Language Models with Language Models (2022, methodology): “Prior work relies on human annotators to generate test cases (Dinan et al., 2019; Nie et al., 2020; Ribeiro et al., 2020; Röttger et al., 2021; Xu et al., 2021b; Wallace et al., 2021) and/or detect failures (Dinan et al.”
Adversarial Training for High-Stakes Reliability (2022, methodology): “Our analysis of the effects of multiple iterations of adversarial training resembles DADC [42].”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “…work has found that DADC leads to improved model robustness on adversarial datasets (Nie et al., 2020; Bartolo et al., 2020), increased sample diversity (Bartolo et al., 2020; Wallace et al., 2021), better training data (Wallace et al., 2021) and better domain generalisation (Bartolo et al., 2021).”
ANLIzing the Adversarial Natural Language Inference Dataset (2020, background): “has since been shown to have training-time benefits above statically collected data (Wallace et al., 2021).”
Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair (2021, result): “057 In contrast to other work focused on adversarial 058 dataset creation for training (Wallace et al., 2021) 059 or training and evaluation data (Le Bras et al., 2020; 060 Nie et al., 2020b), we focus solely on evaluation 061 data, and whether the choice of adversary model 062 can introduce…”