Degrees of freedom and model search
By Robert Tibshirani. 2014
56 citations recorded by the source corpus.
Degrees of freedom is a fundamental concept in statistical modeling, as it provides a quantitative description of the amount of fitting performed by a given procedure. But, despite this fundamental role in statistics, its behavior is not completely well-understood, even in somewhat basic settings. For example, it may seem intuitively obvious that the best subset selection fit with subset size k has degrees of freedom larger than k, but this has not been formally verified, nor has is been precisely studied. At large, the current paper is motivated by this problem, and we derive an exact expression for the degrees of freedom of best subset selection in a restricted setting (orthogonal predictor variables). Along the way, we develop a concept that we name “search degrees of freedom”; intuitively, for adaptive regression procedures that perform variable selection, this is a part of the (total) degrees of freedom that we attribute entirely to the model selection mechanism. Finally, we establish a modest extension of Stein’s formula to cover discontinuous functions, and discuss its potential role in degrees of freedom and search degrees of freedom calculations.
Subfields: Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistics and Probability
- Veeranjaneyulu Sadhanala
- Robert Tibshirani
- Jonathan E. Taylor
- Trevor J. Hastie
- Stefan Wager
- Cynthia Rudin
- Jianqing Fan
- Peter Buhlmann
Citations in the corpus
- Additive models with trend filtering (2017, background): “d by the shrinkage induced by the ‘ 1 penalty in (8) (i.e., for each component t ^ j, there is shrinkage in the dierences between the attained kth derivatives on either side of a selected knot). SeeTibshirani (2015) for a general discussion of this phenomenon. Remark 4 (Easy unbiased degrees of free”