Ridge Regression: Biased Estimation for Nonorthogonal Problems

By A. E. Hoerl, R. Kennard. 2000

12,604 citations recorded by the source corpus.

Subfields: Applied Mathematics, Control and Systems Engineering, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Control and Systems Engineering

In Statistics and Probability

In Optimization & Training

Citations in the corpus

Fast Learning from Distributed Datasets without Entity Matching (2016, methodology): “Here, we consider the Ridge regularized square loss [Hoerl and Kennard, 1970] ( is sym.”
Stealing Hyperparameters in Machine Learning (2018, methodology): “We demonstrate our attacks to popular linear regression algorithms including Ridge Regression (RR) [19] and LASSO [53].”
Is Feature Selection Secure against Training Data Poisoning? (2015, methodology): “…this framework to formalize poisoning attack strategies against popular embedded feature selection methods, including the so-called least absolute shrinkage and selection operator (LASSO) (Tibshirani, 1996), ridge regression (Hoerl & Kennard, 1970), and the elastic net (Zou & Hastie, 2005) (Sect.”
Functional Mechanism: Regression Analysis under Differential Privacy (2012, methodology): “Although regularization is mostly used in regression analysis to avoid overﬁtting [14,29], it also helps achieving a bounded ¯ f D ( ω ) .”
Causal Regularization (2019, background): “It is known [16], for instance, that (3) and (4) maximize the posterior p(a|X̂, Ŷ ) for the priors”
Non-Linear Regression for Bag-of-Words Data via Gaussian Process Latent Variable Set Model (2015, methodology): “Ridge (Hoerl and Kennard 1970), Lasso (Tibshirani 1996) and Elastic net (Zou and Hastie 2005) are standard linear regression models with different regularizers.”
Kernel methods in machine learning (2007, neutral): “• ψ(ξ) = 12ξ 2 yields penalized least squares (LS) regression (Hoerl and Kennard [68], Morozov [102], Tikhonov [136] and Wahba [148]).”
Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates (2013, background): “It is a natural generalization of the ordinary ridge regression estimate (Hoerl and Kennard, 1970) to the non-parametric setting.”
Generalizing from a Few Examples (2019, background): “for new data set. Regularization techniques have been rooted in machine learning, which helps to reduce E est and get better learning performance [80]. Classical examples include Tikhonov regularizer [52], and lasso regularizer [116]. Admittedly, these regularizations can restrict the form of models”
Bayesian group factor analysis with structured sparsity (2016, background): “For example, the well known `2 penalty for coefficients in linear regression models corresponds to Gaussian priors, also known as ridge regression or Tikhonov regularization (Hoerl and Kennard, 1970).”
Multi-task learning for cross-platform siRNA efficacy prediction: an in-silico study (2010, methodology): “More specifically, given two paired sets X i and Y i of n measured values, which could be the error rates evaluated by RMSE for each experiments under the single-task learning model and multi-task learning model in out study, the paired t -test determines whether this two model differ from each othe”
Scaling Up MIMO: Opportunities and Challenges with Very Large Arrays (2012, background): “SHRINKAGE WITH RIDGE PENALTY In ridge regression, the elements of X are shrunk by imposing a penalty on the energy of X [2].”
GWmodel: An R Package for Exploring Spatial Heterogeneity Using Geographically Weighted Models (2013, methodology): “Ridge regression A method to reduce the adverse effects of collinearity in the predictors of a linear model is ridge regression (Hoerl 1962; Hoerl and Kennard 1970).”
Maximin effects in inhomogeneous large-scale data (2014, methodology): “In the general case, the two most interesting choices for the penalty are q = 1, making the estimation lasso-like [Chen, Donoho and Saunders (2001), Tibshirani (1996)], and q = 2 for a ridge-type estimation [Hoerl and Kennard (1970)].”
Moderate Levels of Activation Lead to Forgetting In the Think/No-Think Paradigm (2013, positive): “…in Matlab (Detre et al., 2006; To decode cognitive state information from fMRI data, we trained a ridgeregression model (Hastie, Tibshirani, & Friedman, 2001; Hoerl & Kennard, 1970; for applications of this algorithm to neuroimaging data see, e.g., Newman & Norman, 2010; Poppenk & Norman, 2012).”
Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics (2011, neutral): “Another possibility to determine the regression coefficients is by penalized regression, for instance ridge regression (Hoerl and Kennard, 1970a,b) or the the lasso (Tibshirani, 1996).”