Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization

By Martin Slawski, Matthias Hein. 2012

199 citations recorded by the source corpus.

Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a paradigm according to which sparsity-promoting regularization is regarded as a necessity in such setting. Deviating from this paradigm, we show that non-negativity constraints on the regression coefficients may be similarly effective as explicit regularization if the design matrix has additional properties, which are met in several applications of non-negative least squares (NNLS). We show that for these designs, the performance of NNLS with regard to prediction and estimation is comparable to that of the lasso. We argue further that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in estimation and hence also advantages with respect to support recovery when combined with thresholding. From a practical point of view, NNLS does not depend on a regularization parameter and is hence easier to use.

Subfields: Applied Mathematics, Computational Mechanics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/4616c8d3704aa9e18e3bd5e1c8a909ad0c56cddb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computational Mechanics

In Signal Processing

Citations in the corpus

Nonparametric Shape-Restricted Regression (2017, background): “d coordinate-wise. As the number of generators (or the columns of the design matrix) is increasing with n, we are essentially solving a ‘high-dimensional’ nonnegative least squares problem; see e.g., [129]. A general open question is: Can a theory be developed on the estimation accuracy of the LSE ^”
Sign-constrained least squares estimation for high-dimensional regression (2012, background): “Many similar conditions for success of the Lasso penalization schemes have been derived (for example Zhang and Huang, 2008; Meinshausen and Yu, 2009; Wainwright, 2009; Bunea et al., 2007, 2006; Van De Geer, 2008; Bickel et al., 2009).”
Efficient modeling and forecasting of electricity spot prices (2014, unknown): “Slawski et al. (2013) showed that the non-negative least squares approach is potentially superior to the positive lasso, a lasso approach with a positive parameter constraint.”