SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR
By Peter J. Bickel, Yaacov Ritov, Alexandre B. Tsybakov. 2008
2,646 citations recorded by the source corpus.
We show that, under a sparsity scenario, the Lasso estimator and the Dantzig selector exhibit similar behavior. For both methods, we derive, in parallel, oracle inequalities for the prediction risk in the general nonparametric regression model, as well as bounds on the l p estimation loss for 1 ≤ p ≤ 2 in the linear model when the number of variables can be much larger than the sample size.
Subfields: Statistics and Probability, Statistical Learning Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistics and Probability
- Victor Chernozhukov
- Stefan Wager
- Martin J. Wainwright
- Sara A. van de Geer
- Cun-Hui Zhang
- Peter Buhlmann
- Francis R. Bach
- Robert Tibshirani
In Statistical Learning Theory
- Stefan Wager
- Martin J. Wainwright
- Francis R. Bach
- Tong Zhang
- Victor Chernozhukov
- Bin Yu
- Susan Athey
- Cun-Hui Zhang
Citations in the corpus
- Smoothness, Low Noise and Fast Rates (2010, background): “Bounds similar to (7) have been derived using specialized arguments [12, 30, 5]—here we demonstrate that bounds of these forms can be obtained under simple conditions, using the generic framework we suggest.”
- Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands (2018, background): “Many studies have focused on the lasso and its variants (Bickel, Ritov, and Tsybakov (2009), Belloni, Chernozhukov, and Hansen (2014), Farrell (2015)) and tree/forest based methods (Wager and Athey (2018)).”
- Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, background): “The cone invertibility factor is a generalization of the restricted eigenvalue condition of Bickel et al. (2009), proposed by Ye and Zhang (2010).”
- Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, methodology): “The methods considered by Chan et al. (2015), Graham et al.”
- Geometric median and robust estimation in Banach spaces (2013, background): “1 (Restricted eigenvalue condition, [5]).”
- Optimization with Sparsity-Inducing Penalties (2011, background): “Second, it allows a fruitful theoretical analysis answering fundamental questions related to estimation consistency, prediction efficiency [19, 99] or model consistency [145, 158].”
- Distributed Multi-Task Learning (2015, methodology): “We then define the restricted set C ( | S | , 3) as The following proposition is a simple extension of Theorem 6.2 in [Bickel et al., 2009].”
- Stability selection (2008, methodology): “We illustrate the results on the randomized lasso with a small simulation example: p=n=200 and the predictor variables are sampled from an N .0, Σ/ distribution, where Σ is the identity matrix, except for the entries Σ13 =Σ23 =ρ and their symmetrical counterparts.”
- Sparse coding for multitask and transfer learning (2012, neutral): “cal results guarantee a good performance of this method under the assumption that the vector corresponding to the underlying predictor is sparse, or at least has a very small ℓ1-norm, see for example [11, 12, 31] and references therein. In this work, we consider the case where the predictors are lin”
- Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming (2010, background): “acle rate is achieved up to a factor of √ logn. (Recall that the oracle knows the identity T of significant regressors, and so it can achieve the rate of σ p s/n.) The result above was demonstrated by [2], and closely related results were given in [7], [19], [4], and [16]. [3], [20], [5], [6], and [1”
- Robust Lasso With Missing and Grossly Corrupted Observations (2011, background): “However, this parameter actually controls the sparsity level of the regression vector with respect to the fraction of corruption.”
- On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “, [6], and see [10] which complements the concentration results in [26] for the case of errors with only second moments) for λj K0√log(p)/n [recall that K0 = 1 in the sub-Gaussian case and K0 = K in the (strongly) bounded case] ∥∥X−j (γ̂j − γj )∥∥2n =OP(sjλ2j ), ‖γ̂j − γj‖1 = OP(sjλj ).”
- Double/Debiased Machine Learning for Treatment and Causal Parameters (2016, background): “ o(N 1=2), imposes the (crude) rate requirement " N = o(N 1=4): (3.8) This rate is achievable for many ML methods under structured assumptions on the nuisance parameters. See, among many others, Bickel et al. (2009), Buhlmann and van de Geer (2011), Belloni et al. (2011), Belloni and Chernozhuk”
- Online Learning for Matrix Factorization and Sparse Coding (2009, positive): “We used these patches to create three data sets A , B , and C with increasing patch and dictionary sizes representing various settings which are typical in image processing applications: We have centered and (Bickel et al., 2009), and the constant 1 .”
- On the conditions used to prove oracle results for the Lasso (2009, background): “The restricted eigenvalue condition [2] or the slightly weaker compatibility condition [18] are sufficient for oracle results.”
- PERFORMANCE GUARANTEES FOR INDIVIDUALIZED TREATMENT RULES (2011, background): “See Bickel, Ritov and Tsybakov [3] for other sufficient conditions for (4.8).”