The Benefit of Group Sparsity

By Junzhou Huang, Tong Zhang. 2009

469 citations recorded by the source corpus.

This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is consistent with the data. Moreover, the theory predicts some limitations of the group Lasso formulation that are confirmed by simulation studies.

Subfields: Computational Mechanics, Rheumatology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/66bf80ec240e850dcb78c535698b83490db0a1f7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Rheumatology

In Statistics and Probability

Citations in the corpus

Group Lasso for high dimensional sparse quantile regression models (2011, methodology): “The proof is diﬀerent from that of Huang and Zhang (2010) as they exploited the speciﬁc property of the least squares problem.”
Network Flow Algorithms for Structured Sparsity (2010, unknown): “When these coefficients are organized in groups, a penalty encoding explicitly this prior knowledge can improve the prediction performance and/or interpretability of the learned models [12, 14, 15, 16].”
Direction-of-Arrival Estimation Using a Sparse Representation of Array Covariance Vectors (2011, background): “…note that the -norm of in effect induces the mixed norm for the elements of , which falls into the basic framework of “structured sparsity” or “group lasso” (see, e.g., [11]) that can enforce sparse representations of , as compared with the treatment of considering the -norm of each independently.”
Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations (2013, unknown): “• For the linear model the constants in the group lasso bounds can offset the (logarithmic) suboptimality in rate (Huang and Zhang 2010, Lounici, Pontil, van de Geer, and Tsybakov 2011), and this may be true here as well.”
Group-Sparse Representation With Dictionary Learning for Medical Image Denoising and Fusion (2012, unknown): “Therefore, introducing the intrinsic structure into sparse representation model is a reasonable strategy to improve the performance of sparse representation [19].”
A Selective Review of Group Selection in High-Dimensional Models. (2012, methodology): “This question was addressed by Huang and Zhang (2010), who introduced the concept of strong group sparsity.”
Group Lasso with Overlaps: the Latent Group Lasso approach (2011, unknown): “Keywords: group Lasso, sparsity, graph, support recovery, block regularization, feature selection”
A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers (2009, background): “[39] and Huang and Zhang [27] for this special case of exact block sparsity.”
Optimization with Sparsity-Inducing Penalties (2011, background): “A regularization norm exploiting explicitly this group structure, or `1-group norm, can be shown to improve the prediction performance and/or interpretability of the learned models [61, 83, 106, 116, 141, 156].”
Taking Advantage of Sparsity in Multi-Task Learning (2009, result): “Finally, we note that we recently came across the work [16], whose results are similar to those in Section 3 below.”
Sparse Modeling for Image and Vision Processing (2014, unknown): “2005, Yuan and Lin, 2006, Obozinski et al., 2009, Huang and Zhang, 2010].”
Fast global convergence of gradient methods for high-dimensional statistical recovery (2011, background): “, [45, 24, 19]), and SDP relaxations for various problems, including sparse PCA and low-rank matrix estimation (e.”
High-dimensional regression with unknown variance (2011, methodology): “Under some assumptions on X, Huang and Zhang [44] or Lounici et al.”
Robust Lasso With Missing and Grossly Corrupted Observations (2011, methodology): “If the terms related to the corruption e? are omitted, then we obtain similar parameter estimation bound as in the standard Lasso (e.g. [10], [34]).”
Learning with structured sparsity (2009, background): “For example, in group sparsity, one assumes that variables in the same group tend to be zero or nonzero simultaneously.”
Block-Sparse Signals: Uncertainty Relations and Efficient Recovery (2009, result): “Corresponding numerical results are reported in Section VII.”