Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

By Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S. Anderson, Yaron Singer, Amin Karbasi. Neural Information Processing Systems, 2023

711 citations recorded by the source corpus.

While Large Language Models (LLMs) display versatile functionality, they continue to generate harmful, biased, and toxic content, as demonstrated by the prevalence of human-designed jailbreaks. In this work, we present Tree of Attacks with Pruning (TAP), an automated method for generating jailbreaks that only requires black-box access to the target LLM. TAP utilizes an attacker LLM to iteratively refine candidate (attack) prompts until one of the refined prompts jailbreaks the target. In addition, before sending prompts to the target, TAP assesses them and prunes the ones unlikely to result in jailbreaks, reducing the number of queries sent to the target LLM. In empirical evaluations, we observe that TAP generates prompts that jailbreak state-of-the-art LLMs (including GPT4-Turbo and GPT4o) for more than 80% of the prompts. This significantly improves upon the previous state-of-the-art black-box methods for generating jailbreaks while using a smaller number of queries than them. Furthermore, TAP is also capable of jailbreaking LLMs protected by state-of-the-art guardrails, e.g., LlamaGuard.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/14e8cf5a5e6a7b35e618b08f5cf06f572b3a54e0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Improving Alignment and Robustness with Circuit Breakers (2024, neutral): “Automated red teaming has been shown effective in Perez et al. [53], Chao et al. [11], Mehrotra et al. [41], Zeng et al. [73].”
Many-shot Jailbreaking (2024, methodology): “: We find that MSJ is more effective than the PAIR (Chao et al., 2023) and TAP (Mehrotra et al., 2023), ART (Perez et al., 2022a).”
StruQ: Defending Against Prompt Injection with Structured Queries (2024, neutral): “One example is Tree-of-Attacks with Pruning (TAP) [17].”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, neutral): “• PAIR : Iterative prompting of an attacker LLM to adaptively explore and elicit specific harmful behaviors from the target LLM. • TAP : Tree-structured prompting of an attacker LLM to adaptively explore and elicit specific harmful behaviors from the target LLM. • AutoDAN : A semi-automated method t”
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (2024, neutral): “…considered simple as it does not require gradient information (Zou et al., 2023; Geisler et al., 2024), auxiliary LLMs to iteratively optimize the jailbreaks (Chao et al., 2023; Mehrotra et al., 2023; Zeng et al., 2024), or multi-turn conversations (Cheng et al., 2024; Russi-novich et al., 2024).”
A StrongREJECT for Empty Jailbreaks (2024, neutral): “Many jailbreak papers claim that their jailbreaks can bypass LLM safety training with near-100% attack success rates [43, 18, 40, 4, 19, 29, 39, 7, 17, 21, 44].”
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (2024, background): “…refine hand-crafted jailbreak templates (Yu et al., 2023), translate goal strings into low-resource languages (Yong et al., 2023; Deng et al., 2023), generate jailbreaks (Chao et al., 2023; Mehrotra et al., 2023), or rephrase harmful requests (Shah et al., 2023; Zeng et al., 2024; Takemoto, 2024).”
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, neutral): “Finally, we wrap up our evaluation by assessing the robustness of ICA and ICD in relation to their demonstration selection, highlighting the universality of adversarial demonstrations, and further exploring the interaction between ICA and ICD.”
Jailbreaking Black Box Large Language Models in Twenty Queries (2023, neutral): “We optimize a single adversarial suffix for each behavior.”
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “…names to segmented harmful text (Wu et al., 2024; Kang et al., 2024)), prompt-level (e.g., rewriting user prompts with an LLM (Chao et al., 2023; Mehrotra et al., 2023; Tian et al., 2023; Ge et al., 2023)), language-level (e.g., translating into a language that lacks LLM safety (Qiu et al.,…”