Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming

By Alexandre Belloni, Victor Chernozhukov, Lie Wang. 2010

718 citations recorded by the source corpus.

We propose a pivotal method for estimating high-dimensional sparse linear regression models, where the overall number of regressors p is large, possibly much larger than n, but only s regressors are significant. The method is a modification of the lasso, called the square-root lasso. The method is pivotal in that it neither relies on the knowledge of the standard deviation σ nor does it need to pre-estimate σ. Moreover, the method does not rely on normality or sub-Gaussianity of noise. It achieves near-oracle performance, attaining the convergence rate σl(s/n) log pr-super-1/2 in the prediction norm, and thus matching the performance of the lasso with known σ. These performance results are valid for both Gaussian and non-Gaussian errors, under some mild moment restrictions. We formulate the square-root lasso as a solution to a convex conic programming problem, which allows us to implement the estimator using efficient algorithmic methods, such as interior-point and first-order methods. Copyright 2011, Oxford University Press.

Subfields: Artificial Intelligence, Computational Mechanics, Mathematical Optimization, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Mathematical Optimization

In Signal Processing

Citations in the corpus

Causal inference by using invariant prediction: identification and confidence intervals (2015, background): “variables with non-zero regression coefficient with square-root Lasso estimation [Belloni et al., 2011], Lasso [Tibshirani, 1996] or boosting [Schapire et al.”
Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso (2017, neutral): “…could be considered, for example, `1-penalized alternatives to the lasso, like the Dantzig selector (Candes and Tao, 2007) and square-root lasso (Belloni et al., 2011); greedy alternatives to forward stepwise algorithm, like matching pursuit (Mallat and Zhang, 1993) and orthogonal matching…”
Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, methodology): “Belloni et al. (2014, 2017) show that attempting to control for high-dimensional confounders using a regularized regression adjustment obtained via, e.g., the lasso, can result in substantial biases. Belloni et al. (2014) propose an augmented variable selection scheme to avoid this effect, while Bel”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, methodology): “Scale independent regularization can also be achieved numerically by using the square-root lasso introduced in [3], giving an approximate inverse, ̂√Lasso say, as alternative for ̂Lasso.”
Double/Debiased Machine Learning for Treatment and Causal Parameters (2016, background): “ = o(N 1=4): (3.8) This rate is achievable for many ML methods under structured assumptions on the nuisance parameters. See, among many others, Bickel et al. (2009), Buhlmann and van de Geer (2011), Belloni et al. (2011), Belloni and Chernozhukov (2011), Belloni et al. (2012), and Belloni and Cherno”
Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain (2012, methodology): “In terms of size of 5% level tests, the Lasso-based IV estimator performs comparably to or better 3One exception is the work of Belloni, Chernozhukov, and Wang (2011b) which considers square-root-Lasso estimators and shows that their use allows for pivotal penalty choices.”
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems (2011, background): “If the penalty level is set at λ = σ̂2√(lnp)/n, the Lasso estimator becomes the square-root Lasso [3].”
Model confidence bounds for variable selection (2016, methodology): “For the Laplace distribution, LAD Lasso and SQRT Lasso perform better than the rest, because LAD Lasso and SQRT Lasso are more robust against heavy tail random errors, therefore, offer lower model selection uncertainty.”
An Overview on the Estimation of Large Covariance and Precision Matrices (2015, neutral): “The SQRT-Lasso is a penalized optimization algorithm for solving high dimensional linear regression problems.”
Sign-constrained least squares estimation for high-dimensional regression (2012, background): “Problem (1) is a convex optimization problem and can be solved with general quadratic programming problem solvers, including active set (Lawson and Hanson, 1995), iterative (Kim et al., 2006) and interior-point approaches (Bellavia et al., 2006).”
A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models (2014, methodology): “In an early work, [15] showed that the limiting distribution of the Lasso estimator is nonnormal even in the low dimensional setting.”
Program evaluation with high-dimensional data (2013, methodology): “When U is a singleton the strategy above is similar to Bickel et al. (2009), Belloni and Chernozhukov (2013), and Belloni et al. (2011), who use an analog of (6.4) to derive the properties of Lasso and Post-Lasso.”
Efficient Inference of Average Treatment Effects in High Dimensions via Approximate Residual Balancing (2016, methodology): “tion with the lambda.1se rule from the glmnet package [Friedman et al.,2010]. InBelloni et al.[2014], the authors recommend selecting using more sophisticated methods, such as the square-root lasso [Belloni et al., 2011]. However, in our simulations, our implementation ofBelloni et al.[2014] still a”
Massachusetts Institute of Technology Department of Economics Working Paper Series Sparse Models and Methods for Optimal Instruments with an Application to Eminent Domain Sparse Models and Methods for Optimal Instruments with an Application to Eminent Domain (positive): “Note also that the post-1-penalized procedures have only been analyzed for the case of LASSO and √ LASSO, [4] and [6], but we expect that similar results carry over to other procedures listed above, namely Dantzig and grouped LASSO.”
Goodness‐of‐fit tests for high dimensional linear models (2015, methodology): “Sometimes we will omit the rst argument of ^ for convenience in which case it will always be the true parameter value under the null, . Here we are using the Lasso in the square-root parametrisation [Belloni et al., 2011, Sun and Zhang, 2012] rather than the conventional version where the term in th”
Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression (2018, methodology): “Finally, we mention that apart from the Lasso, there are many other estimators available for high dimensional linear regression, including, for instance, the Dantzig selector (Candes and Tao, 2007) and the square-root Lasso (Belloni et al., 2011), among others.”