DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

By Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn. 2023

1,117 citations recorded by the source corpus.

The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023, methodology): “Mitchell et al. (2023) observed that machine-generated texts tend to lie in the local curvature of the log probability and proposed De-tectGPT, whose prominent performance can only be guaranteed by the large size of the perturbation function and by a large number of perturbations, and thus costs…”
Regulating ChatGPT and other Large Generative AI Models (2023, background): “This should be coupled with research on AI-content detection to highlight such output where watermarks fail [121, 140].”
Defending ChatGPT against jailbreak attack via self-reminders (2023, neutral): “The collection and filtering process for the further 37 jailbreak prompts in the independent training set is detailed in Supplementary Information section 1.2.”
Detecting Pretraining Data from Large Language Models (2023, methodology): “This approach is identical to the DetectGPT (Mitchell et al., 2023) method recently proposed for classifying machine-generated vs. human-written text.”
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, methodology): “Notably, Mitchell et al. (2023) propose DetectGPT, which works similarly to our attack as it compares the likelihood of a given sample under the target model to the likelihood of perturbed samples and hypothesizes that the likelihood of perturbations is smaller than that of texts the model has…”
Data Feedback Loops: Model-driven Amplification of Dataset Biases (2022, background): “Developing more effective filters, such as powerful discriminators to detect between artificially generated and human-created content (Mitchell et al., 2023) is another fruitful direction.”
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, methodology): “(3) Note that · denotes the dot product between two vector-valued quantities.”
LLM Evaluators Recognize and Favor Their Own Generations (2024, positive): “In particular, the detector LLM is not given explicit access to information such as perplexity, which in many detection methods is a crucial component (Mitchell et al., 2023; Hans et al., 2024).”
An Empirical Study of the Non-Determinism of ChatGPT in Code Generation (2023, neutral): “The randomness in LLMs typically comes from the sampling methods used during text generation, such as top-k sampling or nucleus sampling [31, 49].”
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (2023, methodology): “A typical zero-shot classifier, DetectGPT (Mitchell et al., 2023), works under the assumption that machine-generated text variations typically have lower model probability than the original, while human-written ones could go either way.”
Provable Robust Watermarking for AI-Generated Text (2023, positive): “DetectGPT [Mitchell et al., 2023] suggests that AI-generated passages tend to reside in the negative curvature of the log probability of texts.”
Can AI-Generated Text be Reliably Detected? (2023, methodology): “In this section, we discuss paraphrasing attacks that can degrade type-II errors of state-of-the-art AI text detectors such as soft watermarking [1], zero-shot detectors [2], trained neural network-based detectors [3], and retrieval-based detectors [4].”
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (2023, methodology): “They ﬁne-tuned this classiﬁer using generations from 34 language models, with text sourced from Wikipedia, WebText [Radford et al., 2019], and their internal human demonstration data.”
Latent Trajectory Discrimination for AI-Generated Text Detection (2026, unknown): “This process introduces strong local dependencies and is expected to induce locally consistent transitions in the latent representation space [13, 23].”
Human or Machine? A Survey on Machine-Generated Text Detection (2026, unknown): “Reference [89] developed and evaluated several black-box (GPTZero 33 and ZeroGPT), statistical (log-likelihood [101], log-rank, DetectGPT [107], and rank, GLTR [108]), and fine-tuned (mGPT, mBERT, MDeBERTa, GPT-2, ELECTRA and RoBERTa) detectors.”