PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

By Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Hao Chen, Yidong Wang, Linyi Yang, Weirong Ye, Neil Zhenqiang Gong, Yue Zhang, Xingxu Xie. LAMPS@CCS, 2023

312 citations recorded by the source corpus.

The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this vital need, we introduce PromptRobust, a robustness benchmark designed to measure LLMs' resilience to adversarial prompts. This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts, crafted to mimic plausible user errors like typos or synonyms, aim to evaluate how slight deviations can affect LLM outcomes while maintaining semantic integrity. These prompts are then employed in diverse tasks including sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our study generates 4, 788 adversarial prompts, meticulously evaluated over 8 tasks and 13 datasets. Our findings demonstrate that contemporary LLMs are not robust to adversarial prompts. Furthermore, we present comprehensive analysis to understand the mystery behind prompt robustness and its transferability. We then offer insightful robustness analysis, beneficial to both researchers and everyday users. Code is available at https://github.com/microsoft/promptbench.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a2ce9963f1f072d578b1a1f1b995fec75e8c2247

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhixu Li
2. Jindong Wang
3. Xingxu Xie
4. Jiajun Chen
5. Shujian Huang
6. Hao Chen
7. Yidong Wang
8. Linyi Yang

Citations in the corpus

• A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily (2023, methodology): “To further investigate why the generalized attack pattern of ReNeLLM is effective, we conduct attention visualization experiments following the work of Zhu et al. (2023a) to examine the impact of adversarial prompts on LLMs’ attention towards input words.”
• A Survey on Evaluation of Large Language Models (2023, background): “For adversarial robustness, Zhu et al. [249] evaluated the robustness of LLMs to prompts by proposing a unified benchmark called PromptBench.”
• Is ChatGPT a Good NLG Evaluator? A Preliminary Study (2023, background): “(Zhu et al., 2023), and we think this under-explored LLM research direction deserves more research attention.”