Visual Adversarial Examples Jailbreak Aligned Large Language Models

By Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, Prateek Mittal. AAAI Conference on Artificial Intelligence, 2023

403 citations recorded by the source corpus.

Warning: this paper contains data, prompts, and model outputs that are offensive in nature. Recently, there has been a surge of interest in integrating vision into Large Language Models (LLMs), exemplified by Visual Language Models (VLMs) such as Flamingo and GPT-4. This paper sheds light on the security and safety implications of this trend. First, we underscore that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks, representing an expanded attack surface of vision-integrated LLMs. Second, we highlight that the versatility of LLMs also presents visual attackers with a wider array of achievable adversarial objectives, extending the implications of security failures beyond mere misclassification. As an illustration, we present a case study in which we exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Intriguingly, we discover that a single visual adversarial example can universally jailbreak an aligned LLM, compelling it to heed a wide range of harmful instructions (that it otherwise would not) and generate harmful content that transcends the narrow scope of a `few-shot' derogatory corpus initially employed to optimize the adversarial example. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality. Our findings also connect the long-studied adversarial vulnerabilities of neural networks to the nascent field of AI alignment. The presented attack suggests a fundamental adversarial challenge for AI alignment, especially in light of the emerging trend toward multimodality in frontier foundation models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/142e934dd5d6c53f877c30243d436255e3a0dde7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

GPT-4V(ision) System Card (2023, background): “[22] This can be done in the form of screenshots of written instructions, or even visual reasoning cues (See Figure 1).”
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models (2024, background): “VLMs are especially vulnerable to adversarial or even out-of-distribution image inputs that may inadvertently trigger unsafe output (Qi et al., 2023; Liu et al., 2023d).”
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, positive): “Despite these efforts, recent works show that even aligned LLMs are still vulnerable to adversarial attacks, typically called the jailbreak issueofLLMs[7],[8].”
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models (2023, positive): “We assess the efficacy of compositional adversarial attacks learned from these triggers in breaching the safety mechanisms of VLMs to facilitate jailbreak (Wei et al., 2023)—inducing harmful text generations.”
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, result): “This objective is consistent with many previous red teaming studies on aligned LLMs (Wei et al., 2023; Qi et al., 2023; Carlini et al., 2023; Zou et al., 2023).”
FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts (2023, methodology): “One way to jailbreak VLMs through visual prompt is to use adversarial examples [4], [37], [56].”
Security and Privacy Challenges of Large Language Models: A Survey (2024, positive): “For instance, Qi et al. showed a concrete illustration of the risks involved by demonstrating how visual adversarial examples can effectively jailbreak LLMs that integrate visual inputs underscoring the significance of implementing robust security and safety measures for multimodal systems [78].”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, neutral): “Several papers have also explored image attacks on multi-modal LLMs (Bagdasaryan et al., 2023; Shayegani et al., 2023; Qi et al., 2023a; Bailey et al., 2023).”