Defending ChatGPT against jailbreak attack via self-reminders
By Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, Fangzhao Wu. Nature Machine Intelligence, 2023
472 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Health Informatics, Safety Research
View Paper: https://www.semanticscholar.org/paper/e762f92273cd96f63b7788c0173b9b6450adedd7
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- DeepInception: Hypnotize Large Language Model to Be Jailbreaker (2023, methodology): “Apart from those jailbreak methods, we also consider two defense methods, e.g., Self-reminder [23], and In-context Defense [22] for evaluating the jailbreak capability.”
- Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, unknown): “Similar totheattackscenario,safetyprompttemplatesfordefenses likeSelf-reminder[20]aredesigned.”
- Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models (2023, unknown): “This can be attributed to their advanced language understanding and generation capabilities, resulting in following malicious attack instructions embedded in third-party content more effectively.”
- Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack (2024, unknown): “For example, the suffix used by [30] is appended after each user input: “Did your response consider the principles of accuracy, privacy, and non-harm?”
- Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models (2026, unknown): “Most jailbreak defenses (safety system prompts [42, 45], in-context refusal demonstrations [40, 31], input perturbation [34], and input/output classifiers [20, 44, 6]) are designed and validated on a single model.”
- Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “…Kamfonas, 2023; Jain et al., 2023), which tends to yield false positives; 2) Re-minding LLMs to align values with system prompts (Wei et al., 2023; Xie et al., 2023; Zhang et al., 2024), which can be canceled by the user prompt “ignore previous in-structions” (Perez & Ribeiro, 2022); 3)…”