Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation

By Max Bartolo, Tristan Thrush, Robin Jia, Sebastian Riedel, Pontus Stenetorp, Douwe Kiela. 2021

114 citations recorded by the source corpus.

Despite recent progress, state-of-the-art question answering models remain vulnerable to a variety of adversarial attacks. While dynamic adversarial data collection, in which a human annotator tries to write examples that fool a model-in-the-loop, can improve model robustness, this process is expensive which limits the scale of the collected data. In this work, we are the first to use synthetic adversarial data generation to make question answering models more robust to human adversaries. We develop a data generation pipeline that selects source passages, identifies candidate answers, generates questions, then finally filters or re-labels them to improve quality. Using this approach, we amplify a smaller human-written adversarial dataset to a much larger set of synthetic question-answer pairs. By incorporating our synthetic data, we improve the state-of-the-art on the AdversarialQA dataset by 3.7F1 and improve model generalisation on nine of the twelve MRQA datasets. We further conduct a novel human-in-the-loop evaluation and show that our models are considerably more robust to new human-written adversarial examples: crowdworkers can fool our model only 8.8% of the time on average, compared to 17.6% for a model trained without synthetic data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Some of these efforts augment humans (through guidelines, templates, programmatic generation of attacks, and various combinations thereof) to devise test cases that cause systems to fail [45, 46, 29, 21, 30, 55, 6, 23].”
Red Teaming Language Models with Language Models (2022, background): “Bartolo et al. (2021a) learn to generate test cases but using ∼50k manuallywritten examples.”
Adversarial Training for High-Stakes Reliability (2022, background): “There is a large and growing literature on both adversarial attacks and adversarial training for large language models [33, 34, 35, 36, 37].”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, methodology): “Dataset generation Another recent approach leverages language models toward fully automatic dataset creation (Schick and Schütze, 2021; Wu et al., 2022; West et al., 2021; Bartolo et al., 2021a, i.a.).”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, methodology): “This work builds on previous work in adversarial data collection methods for QA (Bartolo et al., 2020), and work investigating the use of question generation models to create synthetic adversarial data to improve QA model robustness (Bartolo et al., 2021).”
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets (2022, background): “We add a consistency ﬁltering step (Lewis et al., 2021; Bartolo et al., 2021) to further improve the quality of the generated dataset.”
Question Answering Infused Pre-training of General-Purpose Contextualized Representations (2021, background): “Bartolo et al. (2021) use generated questions to improve robustness to humanwritten adversarial questions.”
On Evaluating Adversarial Robustness of Large Vision-Language Models (2023, background): “…vulnerability observed in vision tasks, early efforts are devoted to investigating adversarial attacks against visual question answering [6, 11, 37, 38, 43, 77, 89, 97, 105] and image caption [1, 14, 99], with the majority of these efforts focusing on conventional CNN-RNN-based models,…”
Jailbreaking Black Box Large Language Models in Twenty Queries (2023, unknown): “While there have been efforts to automate the generation of prompt-level jailbreaks, these methods require prompt engineering [23], manually-generated test cases [3], or retraining large generative models on objectionable text [4], all of which hinders the widespread deployment of these techniques.”