Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation

By Aparna Elangovan, Jiayuan He, Karin M. Verspoor. 2021

119 citations recorded by the source corpus.

Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). However, the presence of overlap between the train and test datasets can lead to inflated results, inadvertently evaluating the model’s ability to memorize and interpreting it as the ability to generalize. In addition, such data sets may not provide an effective indicator of the performance of these methods in real world scenarios. We identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction, and study them to assess the impact of that leakage on the model’s ability to memorize versus generalize.

Subfields: Natural Language Processing, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation (2022, background): “Many models are pre-trained on web text and hence memo-rize its content (Elangovan et al., 2021; Carlini et al., 2021); therefore, they might answer our problems correctly by simply recalling the solutions seen during pre-training if they were trained on StackOverflow or derivative sites.”
Impact of Pretraining Term Frequencies on Few-Shot Reasoning (2022, background): “Although the overlap between the training and evaluation splits of public datasets and their effect on the generalization of the language models have been studied (Elangovan et al., 2021; Lewis et al., 2020), the effect of the pretraining data has gotten less attention.”
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark (2023, background): “Furthermore, LLMs memorization has been studied on data-leakage scenarios (Elangovan et al., 2021).”
Cut the CARP: Fishing for zero-shot story evaluation (2021, result): “This is key to avoiding overﬁtting due to data leakage, otherwise the model could identify (passage, critique) pairs by matching the quotes (Biderman & Scheirer, 2020; Elangovan et al., 2021).”