Fooling Vision and Language Models Despite Localization and Attention Mechanism

By Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, Dawn Song. 2017

88 citations recorded by the source corpus.

Adversarial attacks are known to succeed on classifiers, but it has been an open question whether more complex vision systems are vulnerable. In this paper, we study adversarial examples for vision and language models, which incorporate natural language understanding and complex structures such as attention, localization, and modular architectures. In particular, we investigate attacks on a dense captioning model and on two visual question answering (VQA) models. Our evaluation shows that we can generate adversarial examples with a high success rate (i.e., > 90%) for these models. Our work sheds new light on understanding adversarial attacks on vision systems which have a language component and shows that attention, bounding box localization, and compositional internal structures are vulnerable to adversarial attacks. These observations will inform future work towards building effective defenses.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Backdooring Convolutional Neural Networks via Targeted Weight Perturbations (2018, background): “The other, more prevalent class of attacks leveraging perturbations that has been studied is referred to as adversarial examples [10, 19, 7, 31, 21, 23, 25, 29].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[228] fool an image captioning framework by treating the generated sentences as individual labels.”
Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering (2020, unknown): “A number of methods [36,43] have investigated adversarial attack on the VQA task.”
On Evaluating Adversarial Robustness of Large Vision-Language Models (2023, background): “…in vision tasks, early efforts are devoted to investigating adversarial attacks against visual question answering [6, 11, 37, 38, 43, 77, 89, 97, 105] and image caption [1, 14, 99], with the majority of these efforts focusing on conventional CNN-RNN-based models, assuming white-box access…”
Auditing large language models: a three-layered approach (2023, background): “Considerable literature already exists within computer science on techniques such as model fooling [155], functional testing [156] and template-based stress-testing [157], which all play important roles during technology audits of LLMs.”
Exact Adversarial Attack to Image Captioning via Structured Output Learning With Latent Variables (2019, background): “An early attempt to attack CNN+RNN based tasks was proposed in [36].”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “logue, CoCoA, Switchboard Dialogue Act, OpenSubtitles Cross-model Optical Character Recognition [72] Hillary Clintons emails Scene Text Recognition [73] SVT, ICDAR 2013, IIIT5K Image Captioning [74], [75] MSCOCO, Visual Genome Visual Question Answering [75] Collected in in [86] Visual-Semantic Embed”
Is Robustness the Cost of Accuracy? - A Comprehensive Study on the Robustness of 18 Deep Image Classification Models (2018, background): “A common approach to evaluate the robustness of DNNs is via adversarial attacks [3,4,5,6,7,8,9,10,11], where imperceptible adversarial examples are crafted to mislead DNNs.”