Model selection and estimation in regression with grouped variables

By Ming Yuan, Yi Lin. 2006

7,897 citations recorded by the source corpus.

Subfields: Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

An Overview of Multi-Task Learning in Deep Neural Networks (2017, neutral): “The latter is also known as group lasso and was ﬁrst proposed by [Yuan and Lin, 2006].”
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2016, background): “By treating the set of variables observed at every location as a group, the use of group Lasso ensured that if a location is selected, all of the climate variables observed at that location will be used as relevant features.”
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (2017, methodology): “For example, to determine ﬁlter dimensions and network depth, previous work [31] has proposed to leverage Group LASSO and residual connections to induce structured sparsity corresponding to smaller ﬁlter dimensions and reduced network depth.”
Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning (2021, positive): “The penalty J1(g) is chosen to be the grouped lasso penalty (Yuan & Lin, 2006), J1(g) = d∑ j=1 ‖αj‖2, (8) where each ‖αj‖2 = √∑K k=1 α 2 jk denotes the `2 norm.”
Generalized Sparse Additive Models (2019, background): “(cid:104) (cid:80) (cid:105) is a group lasso-type penalty [Yuan and Lin, 2006] for feature-wise sparsity, and λ sp a sparsity-related tuning parameter [Ravikumar et al., 2009, Lou et al., 2016, Petersen et al., 2016, Sadhanala and Tibshirani, 2018, Koltchinskii and Yuan, 2010, Raskutti et al.,…”
Fused Lasso Additive Model (2014, methodology): “Here, α provides a trade-oﬀ between encouraging ˆ θ j to be piecewise constant, and inducing sparsity on the entire vector ˆ θ j using the group lasso [Yuan and Lin, 2006].”
Estimating treatment effect heterogeneity in randomized program evaluation (2013, methodology): “, see Zhang et al. (2012) who develop a doubly robust estimator for optimal treatment regimes] and longitudinal data settings where the derivation of optimal dynamic treatment regimes is a frequent goal [e.”
Learning generative models for protein fold families (2011, methodology): “Block regularization is most similar in spirit to the group Lasso [33] and the multi-task Lasso [34].”
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization (2020, methodology): “Several representative model compression methods [37, 4, 38, 29] used the group Lasso-like penalties, which deﬁne the incoming or outgoing weights to a node as groups and achieve structured sparsity within a neural network.”
Regularizing Text Categorization with Clusters of Words (2016, background): “One of the most popular structured regularizers, the group lasso (Yuan and Lin, 2006), was proposed to avoid large L2 norms for groups of weights.”
Fast ConvNets Using Group-Wise Brain Damage (2015, methodology): “A classical way to achieve this is through the use of group-sparsity regularization [47, 41, 24].”
Learning Structured Sparsity in Deep Neural Networks (2016, methodology): “To perform efficient convolution, we explored a fine-grain variant of filter-wise sparsity, namely, 2D-filter-wise sparsity, to spatially enforce group Lasso on each 2D filter of W (l)nl,cl,:,:.”
Regularization Paths for Generalized Linear Models via Coordinate Descent (2010, background): “The grouped lasso (Yuan and Lin 2007; Meier et al. 2008), where variables are included or excluded in groups.”
Learning the Number of Neurons in Deep Networks (2016, neutral): “In short, our approach not only lets us automatically perform model selection, but it also yields networks that, at test time, are more effective, faster and require less memory.”
A Survey on Multi-view Learning (2013, methodology): “(2008, 2010) developed the Composite Kernel Learning (CKL) approach, which extends the multiple kernel learning problem to take into account the group structure among kernels and constructs the relationship with group-LASSO (Yuan and Lin, 2006).”
CAM: Causal Additive Models, high-dimensional order search and penalized regression (2013, methodology): “The latter is very well understood: for example, we can follow the route of hypothesis testing in additive models, or sparse regression can be used for additive models [16, 28, 44].”