Sign-constrained least squares estimation for high-dimensional regression
By Nicolai Meinshausen. 2012
85 citations recorded by the source corpus.
Many regularization schemes for high-dimensional regression have been put forward. Most require the choice of a tuning parameter, using model selection criteria or cross-validation schemes. We show that a simple non-negative or sign-constrained least squares is a very simple and effective regularization technique for a certain class of high-dimensional regression problems. The sign constraint has to be derived via prior knowledge or an initial estimator but no further tuning or cross-validation is necessary. The success depends on conditions that are easy to check in practice. A sufficient condition for our results is that most variables with the same sign constraint are positively correlated. For a sparse optimal predictor, a non-asymptotic bound on the L1-error of the regression coefficients is then proven. Without using any further regularization, the regression vector can be estimated consistently as long as \log(p) s/n -> 0 for n -> \infty, where s is the sparsity of the optimal regression vector, p the number of variables and n sample size. Network tomography is shown to be an application where the necessary conditions for success of non-negative least squares are naturally fulfilled and empirical results confirm the effectiveness of the sign constraint for sparse recovery.
Subfields: Computational Mechanics, Radiology, Nuclear Medicine and Imaging, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/245cd6857cc1516f1b7e42f7d7f9d29e8e8effaf
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Radiology, Nuclear Medicine and Imaging
In Statistics and Probability
Citations in the corpus
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization (2012, background): “A result of the same spirit as Theorem 2 is shown in the recent paper [34] by Meinshausen who has independently studied the performance of NNLS for high-dimensional linear models.”
- Efficient modeling and forecasting of electricity spot prices (2014, unknown): “This sparsity effect in a NNLS problem was recently analyzed by Meinshausen et al. (2013).”
- Recommender Systems (2016, unknown): “Constrained linear regression methods are discussed in [430].”