Group Lasso with Overlaps: the Latent Group Lasso approach

By Guillaume Obozinski, Laurent Jacob, Jean-Philippe Vert. 2011

191 citations recorded by the source corpus.

We study a norm for structured sparsity which leads to sparse linear predictors whose supports are unions of prede ned overlapping groups of variables. We call the obtained formulation latent group Lasso, since it is based on applying the usual group Lasso penalty on a set of latent variables. A detailed analysis of the norm and its properties is presented and we characterize conditions under which the set of groups associated with latent variables are correctly identi ed. We motivate and discuss the delicate choice of weights associated to each group, and illustrate this approach on simulated data and on the problem of breast cancer prognosis from gene expression data.

Subfields: Rheumatology, Statistics and Probability, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Rheumatology

In Statistics and Probability

In Mathematical Optimization

Citations in the corpus

Frank-Wolfe Splitting via Augmented Lagrangian Method (2018, background): “, 2010) or the latent group lasso (Obozinski et al., 2011), the projection step can be a time-consuming operation, while the Frank-Wolfe”
Sparse Partially Linear Additive Models (2014, methodology): “After posting the initial draft of this paper online, we learned of a similar method being developed independently and concurrently to ours (Chouldechova and Hastie, 2015); their approach to the GPLAM problem also makes use of an overlapping group lasso penalty, but uses a different form of penalty ”
Optimization with Sparsity-Inducing Penalties (2011, background): “It should be noted that a careful choice of the weights is much more important in the situation of overlapping groups than in the case of disjoint groups, as it influences possible sparsity patterns [105].”
Sparse Overlapping Sets Lasso for Multitask Learning and its Application to fMRI Analysis (2013, methodology): “As the α G → ∞ the (cid:96) 1 term becomes redundant, reducing h ( x ) to the overlapping group lasso penalty introduced in [6], and studied in [12, 13].”
Combinatorial Penalties: Structure preserved by convex relaxations (2017, methodology): “relaxation of 1qF (supp(w)) + 1 p‖w‖ p p for general set functions and draw connections to the latent group Lasso norm [30].”
Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization (2013, methodology): “In this setting, a slight generalization of the latent group norm (Obozinski et al., 2011) is given by x G := min v (g) ∈R |g| g∈G v (g) g s.t. x = g∈G v [g] .”
Frank-Wolfe with Subsampling Oracle (2018, background): “As a result, for sparse atomic sets such as the ℓ 1 ball, the group lasso ball (also known as ℓ 1 /ℓ 2 ball), or even the latent group lasso (Obozinski et al., 2011) ball, only a few coordinates of the gradient need to be evaluated at each iteration.”
Bayesian Sparse Group Selection (2016, background): “Jacob, Obozinski, and Vert (2009) and Obozinski, Jacob, and Vert (2012) studied group Lasso with overlapping groups, and they suggested representing the coefficient vector, β, as the sum of a set of latent vectors, each of which is supported on one of the groups.”
A Dual Augmented Block Minimization Framework for Learning with Limited Memory (2015, methodology): “There are also a variety of regularizers R(w) employed in different applications, which includes the L2-regularizer R(w) = λ2 ‖w‖ 2 in ridge regression, L1-regularizer R(w) = λ‖w‖1 in Lasso, nuclear-norm R(w) = λ‖w‖∗ in matrix completion, and a family of structured group norms R(w) = λ‖w‖G [11].”
A LASSO FOR HIERARCHICAL INTERACTIONS (2012, methodology): “Our removal of the symmetry constraint in (7) is analogous to the technique of duplicating columns of the design matrix used in the overlap group lasso [Obozinski, Jacob and Vert (2011)].”