Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

By Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, Mohit Iyyer. Neural Information Processing Systems, 2023

572 citations recorded by the source corpus.

The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated text, including those based on watermarking or outlier detection. However, the robustness of these detection algorithms to paraphrases of AI-generated text remains unclear. To stress test these detectors, we build a 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIPPER drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics. To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider. Given a candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within a certain threshold. We empirically verify our defense using a database of 15M generations from a fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% of human-written sequences as AI-generated. We open-source our models, code and data.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/1c13af186d1e177b85ef1ec3fc7b8d33ec314cfd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, methodology): “…Due to the increasing use of tools like ChatGPT as writing assistants, the field of machine-generated text detection has become of high interest within the research community and is being studied extensively et al., 2023; Krishna et al., 2023; Mitchell et al., 2023; Mireshghallah et al., 2023).”
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023, methodology): “Since for zero-shot methods, detection rates are heavily dependent on the threshold when using discriminative statistics, the AUROC metric is commonly used to measure zero-shot detector performance, which considers the range of all possible thresholds (Krishna et al., 2023).”
Testing of detection tools for AI-generated text (2023, methodology): “Our findings are consistent with previously published studies (Gao et al., 2022; Anderson et al., 2023; Demers, 2023; Gewirtz, 2023; Krishna et al., 2023; Pegoraro et al., 2023; van Oijen, 2023; Wang et al., 2023) and substantially differ from what some detection tools for AI-generated text claim…”
Can AI-Generated Text be Reliably Detected? (2023, methodology): “Most machine learning models, including large language models (LLMs), use pseudorandom number generators in one form or another to produce their outputs.”
GPT detectors are biased against non-native English writers (2023, background): “Our investigation into the effectiveness of simple prompts in bypassing GPT detectors, along with recent studies on paraphrasing attacks (Krishna et al., 2023; Sadasivan et al., 2023), raises significant concerns about the reliability of current detection methods.”
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, unknown): “While one approach is to record (Krishna et al., 2023) or watermark all generated text (Kirchenbauer et al., 2023 1.”
Breaking Distortion-free Watermarks in Large Language Models (2025, unknown): “Prior work on watermark scrubbing studies effective watermark removal by paraphrasing [4, 11, 13, 13, 29] or leveraging the LM’s side information [22].”
WaterPool: A Watermark Mitigating Trade-offs among Imperceptibility, Efficacy and Robustness (2024, background): “The statistical pattern of watermarked texts could be vulnerable to potential attacks, including lexical modification or paraphrasing [Krishna et al., 2023, Kirchenbauer et al., 2023b].”