Cross-Validation for Correlated Data

By Assaf Rabinowicz, Saharon Rosset. Journal of the American Statistical Association, 2019

66 citations recorded by the source corpus.

Abstract–K-fold cross-validation (CV) with squared error loss is widely used for evaluating predictive models, especially when strong distributional assumptions cannot be taken. However, CV with squared error loss is not free from distributional assumptions, in particular in cases involving non-iid data. This article analyzes CV for correlated data. We present a criterion for suitability of standard CV in presence of correlations. When this criterion does not hold, we introduce a bias corrected CV estimator which we term that yields an unbiased estimate of prediction error in many settings where standard CV is invalid. We also demonstrate our results numerically, and find that introducing our correction substantially improves both, model evaluation and model selection in simulations and real data studies. Supplementary materials for this article are available online.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/9d06abaad3cc1e3be4b5c32af72f942633162c0c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Cross-validation: what does it estimate and how well does it do it? (2021, methodology): “First, one could adapt nested CV to cases with dependent data, as is done for standard CV (Rabinowicz and Rosset, 2020).”