Do Language Models Plagiarize?

By Jooyoung Lee, Thai Le, Jinghui Chen, Dongwon Lee. 2022

115 citations recorded by the source corpus.

Past literature has illustrated that language models (LMs) often memorize parts of training instances and reproduce them in natural language generation (NLG) processes. However, it is unclear to what extent LMs “reuse” a training corpus. For instance, models can generate paraphrased sentences that are contextually similar to training samples. In this work, therefore, we study three types of plagiarism (i.e., verbatim, paraphrase, and idea) among GPT-2 generated texts, in comparison to its training data, and further analyze the plagiarism patterns of fine-tuned LMs with domain-specific corpora which are extensively used in practice. Our results suggest that (1) three types of plagiarism widely exist in LMs beyond memorization, (2) both size and decoding methods of LMs are strongly associated with the degrees of plagiarism they exhibit, and (3) fine-tuned LMs’ plagiarism patterns vary based on their corpus similarity and homogeneity. Given that a majority of LMs’ training data is scraped from the Web without informing content owners, their reiteration of words, phrases, and even core ideas from training sets into generated texts has ethical implications. Their patterns are likely to exacerbate as both the size of LMs and their training data increase, raising concerns about indiscriminately pursuing larger models with larger training corpora. Plagiarized content can also contain individuals’ personal and sensitive information. These findings overall cast doubt on the practicality of current LMs in mission-critical writing tasks and urge more discussions around the observed phenomena. Data and source code are available at https://github.com/Brit7777/LM-plagiarism.

Subfields: Health Informatics, Safety Research, Large Language Models, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Informatics

Neel Guha

In Safety Research

In Large Language Models

In AI Ethics, Fairness & Society

Citations in the corpus

Foundation Models and Fair Use (2023, background): “Furthermore, others have noted that even when there is no verbatim matching, models can output substantially similar material that could be considered plagiarism (or in our setting, infringement not necessarily covered by fair use) (Lee et al., 2022; Carlini et al., 2022).”
Counterfactual Memorization in Neural Language Models (2021, neutral): “Moreover, memorization of sensitive user information or copyrighted materials in the training data [Carlini et al., 2020, Vyas et al., 2023, Lee et al., 2023] leads to practical concerns in real world applications.”
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (2022, background): “But a number of recent papers have demonstrated that large generative models output memorized content [26, 111, 32, 25, 69] even with deduplication [27].”
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (2023, background): “However, their misuse also introduces concerns—especially regarding fake news (Ahmed et al., 2021), malicious product reviews (Adelani et al., 2020), and plagiarism (Lee et al., 2023).”
Large Language Model Unlearning (2023, result): “We have also seen a large number of recent works that show LLMs can memorize and leak copyright-protected information [Carlini et al., 2021, Wahle et al., 2022, Lee et al., 2023, Liu et al., 2023].”
On Provable Copyright Protection for Generative Models (2023, background): “[Lee et al., 2022a, Tirumala et al., 2022, Carlini et al., 2022] show that memorization becomes worse with model size and data reuse.”