On Evaluating Adversarial Robustness of Large Vision-Language Models

By Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin. 2023

369 citations recorded by the source corpus.

Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses. Our findings provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice. Code is at https://github.com/yunqing-me/AttackVLM.

Subfields: Artificial Intelligence, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Su
2. Haotian Liu
3. Yong Jae Lee
4. Huan Sun
5. Boyuan Zheng
6. Chunyuan Li
7. Yuheng Li
8. Kai Zhang

In Multimodal Learning

1. Chunyuan Li
2. Haotian Liu
3. Yong Jae Lee
4. Yu Su
5. Huan Sun
6. Boyuan Zheng
7. Yuheng Li
8. Saining Xie

In Safety, Robustness & Interpretability

1. Huan Sun
2. Yong Jae Lee
3. Yuheng Li
4. Chunyuan Li

Citations in the corpus

• Improved Baselines with Visual Instruction Tuning (2023, background): “The LLaVA architecture is also lever-aged in different downstream tasks and domains, including region-level [8, 55] and pixel-level [25, 49] understanding, biomedical assistants [30], image generation [5], adversarial studies [6, 58].”
• A survey on multimodal large language models (2023, neutral): “[45] consider safety issues and propose to evaluate the robustness of MLLMs to adversarial attacks.”
• FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts (2023, methodology): “One way to jailbreak VLMs through visual prompt is to use adversarial examples [4], [37], [56].”
• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, neutral): “The evaluation process may involve designing test cases tailored to specific tasks and challenges within the medical field.”
• MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI (2023, background): “These benchmarks cover a wide spectrum of specific skills of LMMs, from Optical Character Recognition (OCR) as seen in the study by [44], to adversarial robustness [84] and hallucination [15, 38], e.g., POPE [35] and HaELM [69].”
• A Survey on Evaluation of Large Language Models (2023, neutral): “[244] evaluated LLMs on visual input and transferred them to other visual-linguistic models, revealing the vulnerability of visual input.”
• Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models (2023, neutral): “The works most closely related to ours are by Aich et al. (2022) and Zhao et al. (2023), both of whom also utilize embedding space attacks.”
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models (2023, neutral): “The third manner focuses on one aspect of MLLMs, such as object hallucination [31] or adversarial robustness [53], which is powerless to comprehensive evaluation.”
• Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents (2026, unknown): “Separately, transfer-based adversarial attacks on vision encoders [88, 35, 22, 82, 50, 41, 76] have been studied in single-turn classification and retrieval settings but have not been applied to the persistent memory pipeline of multimodal AI agents.”