Random Design Analysis of Ridge Regression
By Daniel J. Hsu, Sham M. Kakade, Tong Zhang. 2011
8 citations recorded by the source corpus.
This work gives a simultaneous analysis of both the ordinary least squares estimator and the ridge regression estimator in the random design setting under mild assumptions on the covariate/response distributions. In particular, the analysis provides sharp results on the “out-of-sample” prediction error, as opposed to the “in-sample” (fixed design) error. The analysis also reveals the effect of errors in the estimated covariance structure, as well as the effect of modeling errors, neither of which effects are present in the fixed design setting. The proofs of the main results are based on a simple decomposition lemma combined with concentration inequalities for random vectors and matrices.
Subfields: Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistics and Probability
- Francis R. Bach
- Julien Mairal
- Kenji Fukumizu
- Arthur Gretton
- Rodolphe Jenatton
- Peter Richtárik
- Simon Lacoste-Julien
- Cun-Hui Zhang
Citations in the corpus
- On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions (2015, positive): “…similar analysis for kernel herding (Chen et al., 2010; Bach et al., 2012), an extension to fast rates for non-parametric least-squares regression (Hsu et al., 2014) but with an improved computational complexity, and a study of the consequences of our improved approximation result for online…”