BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
By Jiawen Shi, Pan Zhou, Lichao Sun. 2023
110 citations recorded by the source corpus.
Recently, ChatGPT has gained significant attention in research due to its ability to interact with humans effectively. The core idea behind this model is reinforcement learning (RL) fine-tuning, a new paradigm that allows language models to align with human preferences, i.e., InstructGPT. In this study, we propose BadGPT, the first backdoor attack against RL fine-tuning in language models. By injecting a backdoor into the reward model, the language model can be compromised during the fine-tuning stage. Our initial experiments on movie reviews, i.e., IMDB, demonstrate that an attacker can manipulate the generated text through BadGPT.
Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Su Lin Blodgett
- Alexandra Sasha Luccioni
- Irene Solaiman
- Zeerak Talat
- Dylan Baker
- Margaret Mitchell
- Yacine Jernite
- Noah A. Smith
In Health Informatics
In Safety, Robustness & Interpretability
- Irene Solaiman
- Su Lin Blodgett
- Dylan Baker
- Zeerak Talat
- Marie-Therese Png
- Yacine Jernite
- William Agnew
- Alexandra Sasha Luccioni
Citations in the corpus
- Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “For example, data poisoning, backdoor attacks, and prompt injection attacks can all trick large language models into providing inaccurate information in specific instances [220].”