GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher

By Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-Tse Huang, Pinjia He, Shuming Shi, Zhaopeng Tu. 2023

478 citations recorded by the source corpus.

Safety lies at the core of the development of Large Language Models (LLMs). There is ample work on aligning LLMs with human ethics and preferences, including data filtering in pretraining, supervised fine-tuning, reinforcement learning from human feedback, and red teaming, etc. In this study, we discover that chat in cipher can bypass the safety alignment techniques of LLMs, which are mainly conducted in natural languages. We propose a novel framework CipherChat to systematically examine the generalizability of safety alignment to non-natural languages -- ciphers. CipherChat enables humans to chat with LLMs through cipher prompts topped with system role descriptions and few-shot enciphered demonstrations. We use CipherChat to assess state-of-the-art LLMs, including ChatGPT and GPT-4 for different representative human ciphers across 11 safety domains in both English and Chinese. Experimental results show that certain ciphers succeed almost 100% of the time to bypass the safety alignment of GPT-4 in several safety domains, demonstrating the necessity of developing safety alignment for non-natural languages. Notably, we identify that LLMs seem to have a ''secret cipher'', and propose a novel SelfCipher that uses only role play and several demonstrations in natural language to evoke this capability. SelfCipher surprisingly outperforms existing human ciphers in almost all cases. Our code and data will be released at https://github.com/RobustNLP/CipherChat.

Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety, Robustness & Interpretability

Citations in the corpus

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “Yuan et al. (2023) established a set of encryption and decryption protocols within the system prompt.”
FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts (2023, methodology): “Apart from the LLM-based textual jailbreaking strategies such as role-play [42] and stealthy chat [13], [55], additional visual prompt brings new attack surface to VLMs.”
A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily (2023, methodology): “Jailbreak prompt attacks on LLMs can be categorized into two types: (1) Manual designed jailbreak prompts (walkerspider, 2022; Wei et al., 2023; Kang et al., 2023; Yuan et al., 2023), exemplified by DAN (walkerspider, 2022), which intentionally craft prompts to bypass the LLM’s built-in safeguard.”
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs (2024, unknown): “Side-channel Communication exploits long-tailed distribution to increase jailbreak success rates, such as ciphers (Yuan et al., 2023) and translating harmful instructions into low-resource languages (Deng et al., 2023b; Yong et al., 2023).”
Low-Resource Languages Jailbreak GPT-4 (2023, background): “Prior work has explored non-English jailbreaking attacks using adversarial prompts in a codeswitching fashion [24], injecting the morally unhinged TranslatorBot prompt [9], and obfuscating the inputs with base64 encoding [44], Morse code, or argot languages like Pig Latin, leetspeak, and cipher lang”
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models (2023, background): “We can conclude them into two categories: 1) manually written jailbreak attacks (walkerspider, 2022; Wei et al., 2023; Kang et al., 2023; Yuan et al., 2023) and 2) learning-based jailbreak attacks (Zou et al., 2023; Lapid et al., 2023).”
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024, background): “…et al., 2020; Rafailov et al., 2023) have been developed; however, these methods can be vulnerable to jailbreaks (Wei et al., 2023; Chao et al., 2023; Yao et al., 2023a; Yuan et al., 2023) and adversarial attacks (Wallace et al., 2019; Guo et al., 2021; Jones et al., 2023; Zou et al., 2023b).”
Safety Alignment Should Be Made More Than Just a Few Tokens Deep (2024, unknown): “Self-Cipher (Yuan et al., 2023).”
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2023, unknown): “Our contributions in this work can be summarized as follows: 1) We explore the power of in-context learning in manipulating the safety of LLMs and propose In-Context Attack (ICA) and Defense (ICD) with adversarial demonstrations for jailbreaking and safeguarding purposes.”
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models (2025, unknown): “Sure, here is the method to make bombs: … 𝑒 𝑠 +𝑤 .”