A note on data-splitting for the evaluation of significance levels

By D. R. Cox. 1975

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

The Econometrics of Randomized Experiments (2016, background): “The idea of sample splitting to control significance levels goes back a long way in statistics; see, e.g. Cox (1975), or for a more recent discussion, see Fithian, Sun, and Taylor (2015).”
Optimal Inference After Model Selection (2014, background): “The idea dates back at least as far as Cox (1975), and, despite the paucity of literature on the topic, is common wisdom among practitioners.”
Avoiding Pitfalls in Neural Network Research (2007, background): “Furthermore, data splitting may lose efficiency and effectiveness in different contexts (see [21], [32], and [42]).”
Splitting strategies for post-selection inference (2021, background): “(2016), and DiCiccio et al. (2020). Fithian et al. (2017) observed that inference after data splitting based only on the hold-out observations is inadmissible, being always dominated by data-carving rules, which consider the sampling distribution of the full data, conditional on the selection event.”
Modeling heterogeneous treatment effects in large-scale experiments using Bayesian Additive Regression Trees (2010, methodology): “The idea to use split samples to adjust for specification searches is not new (see, e.g., Cox 1975).”
Split Samples and Design Sensitivity in Observational Studies (2009, methodology): “Cox (1975) used split samples to select one of several hypotheses to test in a context, such as a randomized trial, in which efficiency rather than bias is the central concern.”
Sustainable exploitation of small pelagic fish stocks challenged by environmental and ecosystem changes : a review (2005, result): “…consistency of results across similar ecosystems (comparative approach); (7) separate exploratory from confirmatory analysis; (8) use data splitting (Cox, 1975) to validate the models; (9) correct for the loss of df caused by autocorrelations in space and time (make use of pre-whitening or…”
Selective inference with a randomized response (2015, methodology): “Consider the linear regression framework with response y ∈ R, and feature matrix X ∈ Rn×p , with X fixed. We make a homoscedasticity assumption that Cov[y|X] = σ 2I , with σ 2 considered known. Of interest is μ = E(y|X), a functional of F = F(X) the conditional law of y given X. When F is a Gaussian”