Baseline Defenses for Adversarial Attacks Against Aligned Language Models

By Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein. arXiv.org, 2023

743 citations recorded by the source corpus.

As Large Language Models quickly become ubiquitous, it becomes critical to understand their security vulnerabilities. Recent work shows that text optimizers can produce jailbreaking prompts that bypass moderation and alignment. Drawing from the rich body of work on adversarial machine learning, we approach these attacks with three questions: What threat models are practically useful in this domain? How do baseline defense techniques perform in this new domain? How does LLM security differ from computer vision? We evaluate several baseline defense strategies against leading adversarial attacks on LLMs, discussing the various settings in which each is feasible and effective. Particularly, we look at three types of defenses: detection (perplexity based), input preprocessing (paraphrase and retokenization), and adversarial training. We discuss white-box and gray-box settings and discuss the robustness-performance trade-off for each of the defenses considered. We find that the weakness of existing discrete optimizers for text, combined with the relatively high costs of optimization, makes standard adaptive attacks more challenging for LLMs. Future research will be needed to uncover whether more powerful optimizers can be developed, or whether the strength of filtering and preprocessing defenses is greater in the LLMs domain than it has been in computer vision.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/3e30a7ac4886b28eb50151f58e14a1d698cccd0e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (2024, neutral): “PPL detection (Alon & Kamfonas, 2023; Jain et al., 2023).”
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, positive): “First of all, ICD does not require access or modification to the model, including model parameters [Jain et al., 2023] or internal generation algorithms [Li et al.”
A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily (2023, methodology): “Secondly, learning-based prompt attacks circumvent the manual design process, but the suffixes searched by gradient exhibit semantic meaninglessness, making them easily blocked by perplexity-based defense mechanisms (Jain et al., 2023; Liu et al., 2023b; Zhu et al., 2023b).”
Improving Alignment and Robustness with Circuit Breakers (2024, neutral): “Inference-time defenses, such as perplexity filters [1, 26], x s ∼ D s , x r ∼ D r ▷ Sample Batch Elements 3: Loss to be Optimized 7: end for are effective only against non-adaptive attacks [36], while erase-and-check and SmoothLLM [56] incur high computational costs.”
Formalizing and Benchmarking Prompt Injection Attacks and Defenses (2023, methodology): “Paraphrasing [25]: Paraphrasing was originally designed to paraphrase a prompt to prevent jailbreaking attacks.”
LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked (2023, methodology): “LLM S ELF D EFENSE is relatively simple when compared to existing methods of defending against LLM attacks, as existing methods rely on iterative generation or preprocessing [12, 15].”
Jailbreaking Black Box Large Language Models in Twenty Queries (2023, neutral): “However, token-level attacks can be mitigated via randomization [27] or filtering [14].”
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs (2024, neutral): “Specifically, we focus on mutation-based and detection-based defenses, and deliberately omit perplexity-based methods (Alon and Kamfonas, 2023; Jain et al., 2023), which depend on identifying unusually high perplexity, since our generated PAPs are human-readable with low perplexity in nature.”
Certifying LLM Safety against Adversarial Prompting (2023, background): “For example, Jain et al. (2023) and Alon & Kamfonas (2023) study approaches like perplexity filtering, para-phrasing, and adversarial training to defend against adversarial prompts.”
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models (2023, methodology): “As suggested by Alon & Kamfonas (2023); Jain et al. (2023), we evaluate both our method and the baselines in the context against the defense method, a perplexity defense.”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, positive): “As noted in § 2, the goal of the suffix-based attack described in [18] is to solve the feasibility problem in (2.2), which we reproduce here for ease of exposition: Note that any feasible suffix S ⋆ ∈ A m S will be optimal for the following maximization problem. maximize That is, S ⋆ will result in…”
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation (2023, background): “Further work demonstrates it is possible to automatically discover such inputs, known as adversarial prompts (Wen et al., 2023; Jones et al., 2023; Carlini et al., 2023; Zou et al., 2023; Shen et al., 2023).”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, neutral): “Prior work reported negative results using simpler forms of adversarial training (Jain et al., 2023).”
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (2024, methodology): “Conversely, test-time defenses like SmoothLLM (Robey et al., 2023; Ji et al., 2024) and perplexity filtering (Jain et al., 2023; Alon and Kamfonas, 2023) define wrappers around LLMs to detect potential jailbreaks.”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Jin et al. [121] conducted a comprehensive study on LLMs’ understanding of binary code semantics [122] across different architectures and optimization levels, providing key insights for future research in this area.”
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “…approach to attack LLMs’ safeguards based on text embedding models: to find universal “magic words” (i.e., adversarial suffixes) that would increase or decrease the embedding similarity between any pair of texts so as to mislead the safeguards in classifying within the text embedding space.”