Sparse Estimation with Strongly Correlated Variables using Ordered Weighted L1 Regularization
By Mário A. T. Figueiredo, Robert D. Nowak. 2014
31 citations recorded by the source corpus.
This paper studies ordered weighted L1 (OWL) norm regularization for sparse estimation problems with strongly correlated variables. We prove sufficient conditions for clustering based on the correlation/colinearity of variables using the OWL norm, of which the so-called OSCAR is a particular case. Our results extend previous ones for OSCAR in several ways: for the squared error loss, our conditions hold for the more general OWL norm and under weaker assumptions; we also establish clustering conditions for the absolute error loss, which is, as far as we know, a novel result. Furthermore, we characterize the statistical performance of OWL norm regularization for generative models in which certain clusters of regression variables are strongly (even perfectly) correlated, but variables in different clusters are uncorrelated. We show that if the true p-dimensional signal generating the data involves only s of the clusters, then O(s log p) samples suffice to accurately estimate the signal, regardless of the number of coefficients within the clusters. The estimation of s-sparse signals with completely independent variables requires just as many measurements. In other words, using the OWL we pay no price (in terms of the number of measurements) for the presence of strongly correlated variables.
Subfields: Computational Mechanics, Computer Networks and Communications, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Weijie Su
- E.J. Candes
- Jiaxuan Wang
- Jeeheh Oh
- Jenna Wiens
- Richard Baraniuk
- Justin K. Romberg
- Martin J. Wainwright
In Computer Networks and Communications
- E.J. Candes
- Jiaxuan Wang
- Jeeheh Oh
- Jenna Wiens
- Martin J. Wainwright
- Suhas N. Diggavi
- David L. Donoho
- Prateek Jain
In Statistics and Probability
- Weijie Su
- E.J. Candes
- Jiaxuan Wang
- Jeeheh Oh
- Jenna Wiens
- Richard Nock
- Martin J. Wainwright
- Jinchi Lv
Citations in the corpus
- Learning Credible Models (2017, methodology): “…w ∗ | θ | [ i ] is the i th largest parameter sorted by magnitude; and ⊙ is the training time, Sun et al . explore using features identified as relevant Vapnik et al . explore the theory of learning with privileged information In the linear setting, the method simplifies to l 2 regularization on”
- SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax (2015, background): “We remark that a preprint [33] considers statistical properties of a generalization of OSCAR [17] that coincides with SLOPE.”