On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study

By Divyansh Kaushik, Douwe Kiela, Zachary Chase Lipton, Wen-tau Yih. 2021

38 citations recorded by the source corpus.

In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Researchers hope that models trained on these more challenging datasets will rely less on superficial patterns, and thus be less brittle. However, despite ADC’s intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models. In this paper, we conduct a large-scale controlled study focused on question answering, assigning workers at random to compose questions either (i) adversarially (with a model in the loop); or (ii) in the standard fashion (without a model). Across a variety of models and datasets, we find that models trained on adversarial data usually perform better on other adversarial datasets but worse on a diverse collection of out-of-domain evaluation sets. Finally, we provide a qualitative analysis of adversarial (vs standard) data, identifying key differences and offering guidance for future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Question Answering & Reading Comprehension

Citations in the corpus

WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, neutral): “We find that many examples represent genuinely ambiguous cases rather than careless mislabels, echoing previous findings (Pavlick and Kwiatkowski, 2019). test sets (Kaushik et al., 2021) and decreasing data diversity (Bowman and Dahl, 2021).”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, methodology): “…et al., 2021; Wallace et al., 2021), although the approach has been challenged for not necessarily leading to better generalisation on non-adversarial test sets (Kaushik et al., 2021) and being sensitive to the choice of model that was used in the loop (Bowman and Dahl, 2021; Phang et al., 2021).”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, methodology): “Moreover, it remains unclear 144 whether collecting adversarial or non-adversarial 145 data leads to generally more robust models in the 146 long term (Kaushik et al., 2021).”
On Evaluating Adversarial Robustness of Large Vision-Language Models (2023, background): “…vulnerability observed in vision tasks, early efforts are devoted to investigating adversarial attacks against visual question answering [6, 11, 37, 38, 43, 77, 89, 97, 105] and image caption [1, 14, 99], with the majority of these efforts focusing on conventional CNN-RNN-based models,…”
Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair (2021, methodology): “Using adversarially collected data as training data has been shown to lead to better performance on other adversarial datasets, but worse on out-of-domain datasets (Kaushik et al., 2021; Bowman et al., 2020).”
When Combating Hype, Proceed with Caution (2021, background): “Whether adversarially collected test sets are appropriate for comparing the relative effectiveness of models is a largely orthogonal issue (Bowman and Dahl, 2021; Kaushik et al., 2021).”