DeepInception: Hypnotize Large Language Model to Be Jailbreaker

By Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, Bo Han. arXiv.org, 2023

398 citations recorded by the source corpus.

Large language models (LLMs) have succeeded significantly in various applications but remain susceptible to adversarial jailbreaks that void their safety guardrails. Previous attempts to exploit these vulnerabilities often rely on high-cost computational extrapolations, which may not be practical or efficient. In this paper, inspired by the authority influence demonstrated in the Milgram experiment, we present a lightweight method to take advantage of the LLMs' personification capabilities to construct $\textit{a virtual, nested scene}$, allowing it to realize an adaptive way to escape the usage control in a normal scenario. Empirically, the contents induced by our approach can achieve leading harmfulness rates with previous counterparts and realize a continuous jailbreak in subsequent interactions, which reveals the critical weakness of self-losing on both open-source and closed-source LLMs, $\textit{e.g.}$, Llama-2, Llama-3, GPT-3.5, GPT-4, and GPT-4o. The code and data are available at: https://github.com/tmlr-group/DeepInception.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/99eee47dad469790042b973bc7cb40cb63a660b1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zheng Li
2. James T. Kwok
3. Adrian Weller
4. Olivia Watkins
5. Han Shi
6. L. Yu
7. Jincheng Yu
8. Sam Toyer

Citations in the corpus

• MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (2023, background): “…the rapid development of large language models (LLMs) which emerge as the favored approach for various applications and demonstrate multi-dimensional abilities, including instruction following [7, 38, 55, 67], coding assistance [8, 36, 44, 51], and mathematical problem-solving [14, 28, 43, 79].”
• Security and Privacy Challenges of Large Language Models: A Survey (2024, neutral): “This involves reviewing existing studies in the broad area of security, privacy, LLMs, and their intersections, and notably, highlighting future research directions to design novel evaluation protocols and attack defense mechanisms tailored to the evolving landscape of LLMs.”
• A StrongREJECT for Empty Jailbreaks (2024, background): “We contracted the labelers to label as many forbidden prompt-response pairs as possible until our $2,500 research budget was exhausted.”
• Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, neutral): “Besides, many existing defenses face the bottleneck of over-refusal [47] and computational overhead [43] issues, limiting their practicality for real-world applications.”
• Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “Here’s”’ (Mozes et al., 2023) or “Do anything now” (Mozes et al., 2023)) and implicit templates (e.g., role-playing (Bhardwaj & Poria, 2023; Shah et al., 2023), storytelling (Li et al., 2023b) and virtual scenarios (Li et al., 2023a; Kang et al., 2024; Singh et al., 2023; Du et al., 2023)).”