HIGH-DIMENSIONAL GENERALIZED LINEAR MODELS AND THE LASSO

By Sara A. van de Geer. 2008

782 citations recorded by the source corpus.

We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.

Subfields: Finance, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Finance

In Statistics and Probability

Citations in the corpus

Generalized Sparse Additive Models (2019, background): “(cid:107) (cid:107) This is another common condition in the literature for handling general convex loss functions [see e.g., Negahban et al., 2011, van de Geer, 2008].”
Smoothness, Low Noise and Fast Rates (2010, background): “Bounds similar to (7) have been derived using specialized arguments [12, 30, 5]—here we demonstrate that bounds of these forms can be obtained under simple conditions, using the generic framework we suggest.”
Supplementary Appendix for"Inference on Treatment Effects After Selection Amongst High-Dimensional Controls" (2013, neutral): “In case of the logistic link Λ, Lasso for logistic regression is as defined in van de Geer (2008) and Bach (2010), and the associated post-Lasso estimators are as those defined in Belloni, Chernozhukov, and Wei (2013).”
SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR (2008, methodology): “Modified versions of Lasso estimators (non-quadratic terms and/or penalties slightly different from `1) for nonparametric regression with random design are suggested and studied under prediction loss in [14, 25].”
Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming (2010, neutral): “acle knows the identity T of signiﬁcant regressors, and so it can achieve the rate of σ p s/n.) The result above was demonstrated by [2], and closely related results were given in [7], [19], [4], and [16]. [3], [20], [5], [6], and [18] contain other fundamental results obtained for related problems;”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “For recent advances in the LASSO or LASSO-like estimations of Xβ and β, we refer to Greenshtein and Ritov (2004), Candés and Tao (2007), Bunea, Tsybakov and Wegkamp (2007), van de Geer (2008), Zhang and Huang (2008) and Meinshausen and Yu (2009).”
High-dimensional additive modeling (2008, methodology): “Furthermore, the result implies that with large probability, the estimator selects a sup-set of the active functions, provided that the latter have enough signal (such kind of variable screening results have been established for the lasso in linear and generalized linear models [24, 30]).”
On the conditions used to prove oracle results for the Lasso (2009, background): “Such oracle results have been derived by, among others, [4, 19, 21, 2, 16], and for the related Dantzig selector by [11] and [13].”
Elastic-net regularization in learning theory (2008, background): “Non-asymptotic bounds within the framework of statistical learning have been studied in several papers [25, 8, 28, 37, 24, 39, 44, 26].”
Regression shrinkage and selection via the lasso: a retrospective (2011, neutral): “We should focus on developing tools and understanding their properties, to help us and our collaborators to solve important scientific problems.”
Post-Selection Inference for Generalized Linear Models With Many Controls (2013, background): “The literature on high-dimensional generalized linear models has experienced rapid development [31, 21].”
Preconditioning the Lasso for sign consistency (2015, background): “Penalized generalized linear models are ﬁt by iteratively reweighted least squares, which is equivalent to left multiplying by a diagonal matrix at each iteration; van de Geer (2008) highlights how it is the conditioning of the ﬁnal iteration that matters for sign consistency.”