How Biased is the Apparent Error Rate of a Prediction Rule

By Bradley Efron. 1986

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Additive models with trend filtering (2017, methodology): “4 Degrees of freedom In general, given data Y ∈ R with E(Y ) = η, Cov(Y ) = σ(2)I, and an estimator η̂ of η, recall that we define the effective degrees of freedom of η̂ as [Efron, 1986, Hastie and Tibshirani, 1990]: df(η̂) = 1 σ2 n ∑ i=1 Cov ( η̂(Y ), Y i ) , where η̂(Y ) = (η̂(1)(y), . . . , η̂(Y ”
Fused Lasso Additive Model (2014, background): “Then the degrees of freedom of g is deﬁned as [Efron, 1986, Hastie and Tibshirani, 1990].”
Nearly unbiased variable selection under minimax concave penalty (2010, background): “We refer to Efron (1986) and Meyer and Woodroofe (2000) for more discussions about (5.8) and (5.9).”
Degrees of freedom in lasso problems (2011, methodology): “This result is inspired by the LARS work of Efron et al. (2004), though it is not proved in the LARS paper; see Appendix B of Tibshirani (2011) for a proof.”
Torturing data for the sake of generality: How valid are our regression models? (2000, methodology): “However, this is known to bias the estimate of model-prediction error since the model has been optimized to deal with the “noise” in the data and consequently has lost predictive generality outside the original data (Mosteller & Tukey, 1977; Snee, 1977; Efron, 1986; Verbyla, 1986; Chatfield, 1995).”
Data-driven calibration of linear estimators with minimal penalties (2009, methodology): “Compared to classical estimators of σ(2) , such as the one usually used with Mallows’ CL, Ĉ does not depend on the choice of some model assumed to have almost no bias, which can lead to overestimating σ(2) by an unknown amount [18].”
Assessing transferability of ecological models: an underappreciated aspect of statistical validation (2012, background): “However, in-sample error underestimates true model error, especially for small sample sizes (Efron 1986; Fielding & Bell 1997; Olden & Jackson 2000; Burnham & Anderson 2002; Olden, Jackson & Peres-Neto 2002).”
Bayesian measures of model complexity and fit (2002, background): “…better fit, models are compared by trading off these two quantities and, following early work of Akaike (1973), proposals are often formally based on minimizing a measure of expected loss on a future replicate data set: see, for example, Efron (1986), Ripley (1996) and Burnham and Anderson (1998).”
Constrain to perform: Regularization of habitat models (2006, background): “(2)) (Efron, 1986).”
To Log or Not to Log: Bootstrap as an Alternative to the Parametric Estimation of Moderation Effects in the Presence of Skewed Dependent Variables (2000, unknown): “Studies examining similarities in results obtained from bootstrap and normal theory approaches when parametric assumptions are met test the bootstrap’s ability to estimate true latent population distributions (e.g., Diaconis & Efron, 1983; Efron, 1985, 1986; Lunneborg, 1985).”
PENALIZED LINEAR UNBIASED SELECTION (2007, background): “We refer to Efron (1986) and Meyer and Woodroofe (2000) for more discussions about (4.10) and (4.11).”
CHARACTERIZING THE SOLUTION PATH OF MULTICATEGORY SUPPORT VECTOR MACHINES (2006, methodology): “General approaches to tuning, equivalently model selection, can be found in, for example, Efron (1986) and Wahba (1990). Just for illustration of the feasibility of tuning, 100 new data sets of size 300 were generated as random replicates of a tuning set, and λ was chosen to minimize the error rate ”