A Watermark for Large Language Models

By John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein. 2023

964 citations recorded by the source corpus.

Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient open-source algorithm without access to the language model API or parameters. The watermark works by selecting a randomized set of"green"tokens before a word is generated, and then softly promoting use of green tokens during sampling. We propose a statistical test for detecting the watermark with interpretable p-values, and derive an information-theoretic framework for analyzing the sensitivity of the watermark. We test the watermark using a multi-billion parameter model from the Open Pretrained Transformer (OPT) family, and discuss robustness and security.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Furthermore, current research efforts towards watermarking text remain brittle and the area of developing watermarks for machine generated outputs is an active research area [128].”
The Gradient of Generative AI Release: Methods and Considerations (2023, neutral): “Research strives ensure these watermarks are invisible to the human-eye [107], do not affect output quality [46], and tamper-proof from model attacks and alterations like fine-tuning via methods such as embedding noise as watermarks [106].”
GPT detectors are biased against non-native English writers (2023, neutral): “, 2023) and watermarking techniques (Kirchenbauer et al., 2023; Gu et al., 2022).”
Self-Consuming Generative Models Go MAD (2023, methodology): “of watermarking of data generated by LLMs [75] and novel methods for watermarking LLMs [76], diffusion models [77–80], andn GANs [81] are currently active areas of research.”
Exploring AI Ethics of ChatGPT: A Diagnostic Analysis (2023, background): “Existing strategies [81, 82] have demonstrated the effectiveness of preventing the LLM abuse, though they can be invalid via attacks [83].”
Can AI-Generated Text be Reliably Detected? (2023, methodology): “In this section, we discuss paraphrasing attacks that can degrade type-II errors of state-of-the-art AI text detectors such as soft watermarking [1], zero-shot detectors [2], trained neural network-based detectors [3], and retrieval-based detectors [4].”
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, neutral): “While one approach is to record (Krishna et al., 2023) or watermark all generated text (Kirchenbauer et al., 2023 1.”
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (2023, background): “Another line of detection methodology is watermarking that deliberately embeds information within machine-generated text to trace its origin (Jalil & Mirza, 2009; Kamaruddin et al., 2018; Abdelnabi & Fritz, 2021; Gu et al., 2022; Kirchenbauer et al., 2023).”
Data Feedback Loops: Model-driven Amplification of Dataset Biases (2022, methodology): “Watermarking model outputs (Kirchenbauer et al., 2023) is one strategy for avoiding feedback from previous model-labeled samples.”
Auditing large language models: a three-layered approach (2023, methodology): “, through a structured use of model cards [82], datasheets [83], system cards [84], and the watermarking of system outputs [85].”
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023, methodology): “Grinbaum and Ado-maitis (2022) and Abdelnabi and Fritz (2021) watermarked machine-generated text using syntax tree manipulation, while Kirchenbauer et al. (2023) required access to the LLM’s logits at each time step to add the watermark.”
A Categorical Archive of ChatGPT Failures (2023, neutral): “watermarks [30]) in the generated text!”
Regulating ChatGPT and other Large Generative AI Models (2023, background): “Importantly, more interdisciplinary research is necessary to develop markings that are easy to use and recognize, but hard to remove by average users [139].”
Provable Robust Watermarking for AI-Generated Text (2023, positive): “As a baseline of comparison, we also obtain new robustness guarantees for the soft watermarking method proposed in (Kirchenbauer et al., 2023).”
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (2023, methodology): “More formally, let p and q be aligned paragraphs, where p 1 , p 2 ...p N denote sentences of p and q 1 , q 2 , ...q M q1 : My p4 : I don’t know how long I stayed in that position or how long I might have stayed, but apparently I was a very touching sight to my companion and to the two nuns when…”