On Model Selection Consistency of Lasso

By Peng Zhao, Bin Yu. 2006

2,957 citations recorded by the source corpus.

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Multi-Fidelity Automatic Hyper-Parameter Tuning via Transfer Series Expansion (2019, background): “Model configuration is often considered as model selection (Brazdil, Soares, and Da Costa 2003; Chapelle, Vapnik, and Bengio 2002; Maron and Moore 1994; Zhao and Yu 2006) and hyper-parameter tunning (Bergstra and Bengio 2012; Snoek, Larochelle, and Adams 2012).”
Forward stability and model path selection (2021, neutral): “…on assessing consistency and establishing conditions under which it can be guaranteed that the estimated tuning parameter(s) cor-respondingtothechosen ˆ f m convergestotheirtrueoroptimal values [see e.g. Zhao and Yu (2006); Bach (2008); Zhang (2009) or Fan and Lv (2010) for a thorough overview].”
Spectral Regularization Algorithms for Learning Large Incomplete Matrices (2010, background): “We argued that in many problems l1 regularization gives better prediction accuracy [ ZY06 ].”
CAM: Causal Additive Models, high-dimensional order search and penalized regression (2013, background): “The former is a well understood subject thanks to the broad literature in sparse regression and related techniques [17, 36, 41, 44, 47, 48], cf.”
Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm (2005, methodology): “However, the Lasso approach can be inconsistent, even with fixed dimension p, as discussed in detail in Zhao and Yu (2006).”
Sparse additive models (2007, background): “Here, supp(β) = {j : βj 6= 0}. References include Wainwright (2006), Meinshausen and P. B¨uhlmann (2006), Zou (2005) and Zhao and Yu (2007) .”
Optimization with Sparsity-Inducing Penalties (2011, background): “Second, it allows a fruitful theoretical analysis answering fundamental questions related to estimation consistency, prediction efficiency [19, 99] or model consistency [145, 158].”
Stability selection (2008, methodology): “For the lasso, it amounts to the ‘neighbourhood stability’ condition (Meinshausen and Bühlmann, 2006), which is equivalent to the ‘irrepresentable condition’ (Zhao and Yu, 2006; Zou, 2006; Yuan and Lin, 2007).”
Methods and Models for Interpretable Linear Classification (2014, background): “the one that minimizes the number of non-zero coefficients) under very restrictive conditions that are rarely satisfied in practice [119, 70].”
Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming (2010, neutral): “the identity T of signiﬁcant regressors, and so it can achieve the rate of σ p s/n.) The result above was demonstrated by [2], and closely related results were given in [7], [19], [4], and [16]. [3], [20], [5], [6], and [18] contain other fundamental results obtained for related problems; please see”
Nearly unbiased variable selection under minimax concave penalty (2010, background): “Under a strong irrepresentable condition on the normalized Gram matrix Σ≡X′X/n, Meinshausen and Buhlmann (2006), Tropp (2006), Zhao and Yu (2006) and Wainwright (2006) proved that the LASSO is variable selection consistent P{Â=Ao}→ 1 with Â≡ {j : β̂j 6= 0},(1.4) provided that minβj 6=0 |βj |/λ is…”
Robust Lasso With Missing and Grossly Corrupted Observations (2011, background): “3) At the first glance, the parameter γ does not seem to have any meaningful interpretation and the setting γ = 1 seems to be the best selection due to the smallest estimation error it can produce.”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “The vast majority of work has been pursued for point estimation such as consistency for prediction [7, 21], oracle inequalities and estimation of a highdimensional parameter [6, 11, 12, 24, 33, 34, 47, 51] or variable selection [17, 30, 49, 53].”
Learning with Submodular Functions: A Convex Optimization Perspective (2011, background): “The last proposition shows that we get almost surely stable sparsity patterns; in [7], in context where the data y are generated as sparse linear combinations of columns of X with additional noise, sufficient conditions for the recovery of the support are derived, potentially in high-dimensional set”
High-dimensional Ising model selection using ℓ1-regularized logistic regression (2010, methodology): “ graphical models of the covariance matrix E[XXT ] of Gaussian graphical models, and indeed our assumptions are analogous to those imposed in previous work on the Lasso for Gaussian linear regression [23, 32, 39]. In the following we write simply Q∗ for the matrix Q∗ r where the reference node r sho”