LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked
By Alec Helbling, Mansi Phute, Matthew Hull, Sebastian Szyller, Duen Horng Chau. Tiny Papers @ ICLR, 2023
324 citations recorded by the source corpus.
Large language models (LLMs) are popular for high-quality text generation but can produce harmful content, even when aligned with human values through reinforcement learning. Adversarial prompts can bypass their safety measures. We propose LLM Self Defense, a simple approach to defend against these attacks by having an LLM screen the induced responses. Our method does not require any fine-tuning, input preprocessing, or iterative output generation. Instead, we incorporate the generated content into a pre-defined prompt and employ another instance of an LLM to analyze the text and predict whether it is harmful. We test LLM Self Defense on GPT 3.5 and Llama 2, two of the current most prominent LLMs against various types of attacks, such as forcefully inducing affirmative responses to prompts and prompt engineering attacks. Notably, LLM Self Defense succeeds in reducing the attack success rate to virtually 0 using both GPT 3.5 and Llama 2. The code is publicly available at https://github.com/poloclub/llm-self-defense
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/2cdd5c3dc42c0df40bc8839709869af3560d4bfe
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Improving Alignment and Robustness with Circuit Breakers (2024, unknown): “System-level defenses against unsafe inputs or outputs [20, 25, 27] can still be circumvented by sophisticated adversaries [39].”
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Jin et al. [121] conducted a comprehensive study on LLMs’ understanding of binary code semantics [122] across different architectures and optimization levels, providing key insights for future research in this area.”
- From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, unknown): “(Phute et al., 2023) use the same LLM to evaluate its responses as harmful, and provide insights and best practices for GPT-3.”
- Visual Adversarial Examples Jailbreak Aligned Large Language Models (2023, methodology): “Another trend is post-processing model outputs with another LLM optimized for content moderation (Helbling et al. 2023; Weng, Goel, and Vallone 2023).”
- Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models (2026, unknown): “Most jailbreak defenses (safety system prompts [42, 45], in-context refusal demonstrations [40, 31], input perturbation [34], and input/output classifiers [20, 44, 6]) are designed and validated on a single model.”
- Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “…security systems: 1) rule-based matching, with the same drawbacks as it on the input side; 2) another LLM to answer whether the output is harmful (Phute et al., 2023; Inan et al., 2023; Wang et al., 2023), which doubles the cost; 3) alternatively, text classifiers to do this (He et al., 2021;…”