Persistence in high-dimensional linear predictor selection and the virtue of overparametrization

By Eitan Greenshtein, Yaacov Ritov. 2004

379 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Sparse additive models (2007, methodology): “Following Greenshtein and Ritov (2004) we say that an estimator m̂n is persistent (risk consistent) relative to a class of functionsMn, if (58) R(m̂n)−R(m∗n) P→ 0 where (59) m∗n = argmin v∈Mn R(v) is the predictive oracle.”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “For recent advances in the LASSO or LASSO-like estimations of Xβ and β, we refer to Greenshtein and Ritov (2004), Candés and Tao (2007), Bunea, Tsybakov and Wegkamp (2007), van de Geer (2008), Zhang and Huang (2008) and Meinshausen and Yu (2009).”
High-dimensional graphs and variable selection with the Lasso (2006, methodology): “The closest to our setting isthe recentwork ofG reenshtein and Ritov [8],who study consistentprediction in a triangular setup very sim ilar to ours (see also [10]).”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “The vast majority of work has been pursued for point estimation such as consistency for prediction [7, 21], oracle inequalities and estimation of a highdimensional parameter [6, 11, 12, 24, 33, 34, 47, 51] or variable selection [17, 30, 49, 53].”
High-dimensional additive modeling (2008, background): “Substantial progress has been achieved over the last years in estimating high-dimensional linear or generalized linear models where the number of covariates p is much larger than sample size n. The theoretical properties of penalization approaches like the Lasso [28] are now well understood [ 14 , 2”
Vast Portfolio Selection with Gross-exposure Constraints (2012, neutral): “The concept of risk approximation is similar to persistent (Greenshtein and Ritov 2004).”
Statistical significance in high-dimensional linear models (2012, neutral): “Greenshtein and Ritov (2004) proved the ﬁrst result closely related to prediction as measured in (1.3).”
SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax (2015, methodology): “This is already apparent when scanning the literature on penalized sparse estimation procedures such as the Lasso, SCAD [33], the Dantzig selector [24] and MC+ [58], see e.g. [37, 23, 59, 22, 11, 52, 55, 44, 57, 7, 29] for a highly incomplete list of references.”
Sure independence screening in generalized linear models with NP-dimensionality (2009, methodology): “Hall and Miller (2009) proposed a generalized correlation ranking, which allows nonlinear regression.”
A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, background): “The latter property is called persistence in Greenshtein and Ritov (2004) and Greenshtein (2006), and arises frequently in machine learning problems such as document classiﬁcation and computer vision.”
SpAM: Sparse Additive Models (2007, background): “The good empirical success of this estimator has been recently backed up by results confirming that it has strong theoretical properties; see (Greenshtein and Ritov, 2004; Zhao and Yu, 2007; Meinshausen and Yu, 2006; Wainwright, 2006).”
Minimax Rates of Estimation for High-Dimensional Linear Regression Over q -Balls (2009, background): “Various authors have obtained convergence rates for different error metrics, including # 2-error [6, 4, 37], prediction loss [4, 16 ], as well as model selection consistency [37, 25, 33, 38].”
A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers (2009, background): “, [11, 25, 72, 79]), consistency of the parameter estimates in 2 or some other norm (e.”
Compressed and Privacy-Sensitive Sparse Regression (2009, background): “The result of [3] implies that the excess risk converges to zero at the rate”
Assumptionless consistency of the Lasso (2013, background): “Incidentally, the above theorem is related to the notion of persistence defined in [17] and thoroughly investigated in [1].”