Time Travel in LLMs: Tracing Data Contamination in Large Language Models

By Shahriar Golchin, Mihai Surdeanu. 2023

194 citations recorded by the source corpus.

Data contamination, i.e., the presence of test data from downstream tasks in the training data of large language models (LLMs), is a potential major issue in measuring LLMs' real effectiveness on other tasks. We propose a straightforward yet effective method for identifying data contamination within LLMs. At its core, our approach starts by identifying potential contamination at the instance level; using this information, our approach then assesses wider contamination at the partition level. To estimate contamination of individual instances, we employ"guided instruction:"a prompt consisting of the dataset name, partition type, and the random-length initial segment of a reference instance, asking the LLM to complete it. An instance is flagged as contaminated if the LLM's output either exactly or nearly matches the latter segment of the reference. To understand if an entire partition is contaminated, we propose two ideas. The first idea marks a dataset partition as contaminated if the average overlap score with the reference instances (as measured by ROUGE-L or BLEURT) is statistically significantly better with the completions from guided instruction compared to a"general instruction"that does not include the dataset and partition name. The second idea marks a dataset partition as contaminated if a classifier based on GPT-4 with few-shot in-context learning prompt marks multiple generated completions as exact/near-exact matches of the corresponding reference instances. Our best method achieves an accuracy between 92% and 100% in detecting if an LLM is contaminated with seven datasets, containing train and test/validation partitions, when contrasted with manual evaluation by human experts. Further, our findings indicate that GPT-4 is contaminated with AG News, WNLI, and XSum datasets.

Subfields: Artificial Intelligence, Large Language Models, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Evaluation & Benchmarks

Citations in the corpus

LiveBench: A Challenging, Contamination-Free LLM Benchmark (2024, background): “If the LLM has seen the questions of a benchmark during training, its performance on that benchmark will be artificially inflated [15, 17, 24, 47], hence making many LLM benchmarks unreliable.”
Qwen2 Technical Report (2024, background): “For large language models, what counts as contamination and how to run contamination analysis remain an active area of research (Ravaut et al., 2024; Golchin & Surdeanu, 2024; Sainz et al., 2023).”
Proving Test Set Contamination in Black Box Language Models (2023, background): “2019; Mattern et al., 2023) as well as provide some evidence for test set contamination (Sainz et al., 2023; Golchin & Surdeanu, 2023).”
Detecting Pretraining Data from Large Language Models (2023, background): “Golchin & Surdeanu (2023) extends this framework to extract more memorized instances by incorporating partial instance content into the prompt.”