How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

By Yi Zeng, Hong Yi Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, Weiyan Shi. Annual Meeting of the Association for Computational Linguistics, 2024

673 citations recorded by the source corpus.

Most traditional AI safety research has approached AI models as machines and centered on algorithm-focused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. This paper introduces a new perspective to jailbreak LLMs as human-like communicators, to explore this overlooked intersection between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate interpretable persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak performance across all risk categories: PAP consistently achieves an attack success rate of over $92\%$ on Llama 2-7b Chat, GPT-3.5, and GPT-4 in $10$ trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP and, found a significant gap in existing defenses, and advocate for more fundamental mitigation for highly interactive LLMs

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/732ce53c573475f2691a7cfc716cf4f568d17360

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Matthias Hein
2. Maksym Andriushchenko
3. Francesco Croce
4. Dan Hendrycks
5. Vikash Sehwag
6. Mantas Mazeika
7. Steven Basart
8. Nicolas Flammarion

In Safety Research

1. George Pappas
2. Zeming Wei
3. Eric Wong
4. Yisen Wang
5. Yifei Wang
6. Hamed Hassani
7. Alexander Robey

In Sociology and Political Science

1. George Pappas
2. Dan Hendrycks

Citations in the corpus

• Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, positive): “Optimization-based attacks iteratively refine a harmful prompt with query or gradient heuristics to elicit the LLMs to generate harmful content [11], [12], [13], [14], yet often facing the efficiency bottleneck.”
• GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher (2023, neutral): “…2022; Perez et al., 2022; Perez & Ribeiro, 2022; Kang et al., 2023; Li et al., 2023; Ganguli et al., 2022; Schulhoff et al., 2023; OpenAI, 2023b; Jones et al., 2023; Zou et al., 2023; Huang et al., 2024; Zeng et al., 2024; Yu et al., 2023; Liu et al., 2024; Wang et al., 2023a; Deng et al., 2024).”
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (2024, methodology): “…refine hand-crafted jailbreak templates (Yu et al., 2023), translate goal strings into low-resource languages (Yong et al., 2023; Deng et al., 2023), generate jailbreaks (Chao et al., 2023; Mehrotra et al., 2023), or rephrase harmful requests (Shah et al., 2023; Zeng et al., 2024; Takemoto, 2024).”
• Improving Alignment and Robustness with Circuit Breakers (2024, neutral): “These attacks include gradient-based optimization (GCG [78]), LLM optimizers (PAIR [11]), custom jailbreaking pipelines (TAP-Transfer [73], AutoDAN [36], and HumanJailbreaks [62]).”
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (2024, neutral): “…considered simple as it does not require gradient information (Zou et al., 2023; Geisler et al., 2024), auxiliary LLMs to iteratively optimize the jailbreaks (Chao et al., 2023; Mehrotra et al., 2023; Zeng et al., 2024), or multi-turn conversations (Cheng et al., 2024; Russi-novich et al., 2024).”
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, positive): “These are then evolved using a hierarchical genetic algorithm to elicit specific behaviors from the target LLM. • PAP : Adapting requests to do behaviors with a set of persuasive strategies.”
• A StrongREJECT for Empty Jailbreaks (2024, positive): “Persuasive Adversarial Prompts (PAP) instructs an attacker model to persuade a victim model to give it harmful information using techniques like misrepresentation and logical appeals [41].”
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep (2024, neutral): “Interpretation of Our Objective.”