A Non-Parametric Test to Detect Data-Copying in Generative Models

By Casey Meehan, Kamalika Chaudhuri, Sanjoy Dasgupta. 2020

76 citations recorded by the source corpus.

Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} -- where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample non-parametric test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets. For code \& examples, visit this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, background): “If target text occurs in the pretraining corpus, the model can learn to copy the text instead of actually solving the task (Meehan et al., 2020; Carlini et al., 2020).”
Privacy in Deep Learning: A Survey (2020, background): “Deep neural networks have been shown to memorize the training data [52, 53, 54], rather than learning the latent properties of the data, which means they often tend to over-fit to the training data.”
On Memorization in Probabilistic Deep Generative Models (2021, background): “Moreover, hypothesis tests and evaluation metrics have been proposed that measure memorization using distances between observations and model samples [26, 27].”
Documenting the English Colossal Clean Crawled Corpus (2021, unknown): “Thus, if target text occurs in the pretraining corpus, the model can learn to copy the text instead of actually solving the task (Meehan et al., 2020; Carlini et al., 2020), and we measure the model’s data-copying abilities rather than their abilities to solve the end-task.”
Machine Learning for Synthetic Data Generation: a Review (2023, background): “Studies have shown that a sequence that appears multiple times in the training data is more likely to be generated than a sequence that occurred only once [154], [191], [192].”
Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models (2022, unknown): “Related work.”
How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models (2021, background): “Unlike discriminative models for which generalization is easily tested via held-out data, evaluating generalization in generative models is not straightforward (Adlam et al., 2019; Meehan et al., 2020).”
Pros and Cons of GAN Evaluation Measures: New Developments (2021, background): “[38] formalize a notion of overfitting called data-copying where a generative model memorizes the training samples or their small variations (Fig.”
Do Language Models Plagiarize? (2022, background): “ing results, a growing body of literature gave rise to privacy violations of a neural language model resulting from data memorization (Carlini et al., 2019; Thakkar et al., 2021; Truex et al., 2018; Arpit et al., 2017; Meehan et al., 2020).”