LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?

By David Glukhov, Ilia Shumailov, Yarin Gal, Nicolas Papernot, Vardan Papyan. 2023

81 citations recorded by the source corpus.

Large language models (LLMs) have exhibited impressive capabilities in comprehending complex instructions. However, their blind adherence to provided instructions has led to concerns regarding risks of malicious use. Existing defence mechanisms, such as model fine-tuning or output censorship using LLMs, have proven to be fallible, as LLMs can still generate problematic responses. Commonly employed censorship approaches treat the issue as a machine learning problem and rely on another LM to detect undesirable content in LLM outputs. In this paper, we present the theoretical limitations of such semantic censorship approaches. Specifically, we demonstrate that semantic censorship can be perceived as an undecidable problem, highlighting the inherent challenges in censorship that arise due to LLMs' programmatic and instruction-following capabilities. Furthermore, we argue that the challenges extend beyond semantic censorship, as knowledgeable attackers can reconstruct impermissible outputs from a collection of permissible ones. As a result, we propose that the problem of censorship needs to be reevaluated; it should be treated as a security problem which warrants the adaptation of security-based approaches to mitigate potential risks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Gabriel Synnaeve
2. Yossi Adi
3. Jade Copet
4. Nicolas Usunier
5. Tal Remez
6. Hugo Touvron
7. Jianfeng Gao
8. Itai Gat

In Safety, Robustness & Interpretability

1. Nicolas Usunier
2. Ivan Evtimov
3. Gabriel Synnaeve
4. Yossi Adi

Citations in the corpus

• Code Llama: Open Foundation Models for Code (2023, background): “Dual intent prompts are requests for help with writing code that could be used maliciously but the prompt does not directly address the topic (example “Mosaic Prompts” Glukhov et al. (2023)).”
• The application of large language models in medicine: A scoping review (2024, unknown): “103,104 This means that LLMs can recognize and utilize complex patterns and associations that MLS might overlook, thereby providing more accurate predictions in certain cases.”
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, unknown): “The most widely-used defense in production is filtering, but Glukhov et al. (2023) note that output filtering can be foiled if jailbroken LLMs assist malicious users with bypassing detection, e.g., by generating encoded outputs.”
• StruQ: Defending Against Prompt Injection with Structured Queries (2024, unknown): “A possible direction is to use access control and rate-limiting to detect and ban iterative attackers, as suggested by Glukhov et al. [67].”