Discussion: One-step sparse estimates in nonconcave penalized likelihood models

By Cun-Hui Zhang. 2008

61 citations recorded by the source corpus.

Penalized methods are commonly used for selecting variables and fitting high-dimensional data. It is well known that the LASSO is biased and thus cannot attain the estimation efficiency of the oracle selector. Recent studies [6, 7, 8, 10, 11] showed that due to the interference of the bias, the LASSO requires quite strong conditions for consistent variable selection. Since the l1 penalty has the smallest bias among all convex penalty functions with selection features, these studies naturally draw our attention to methodologies based on concave penalties, or equivalently, nonconcave penalized likelihood. Frank and Friedman [5] considered the l� penalty for general α ≥ 0, which is strictly concave for α < 1. Their main interest was to use α as a hyperparameter to “bridge” between the subset selection with α = 0 and the ridge regression with α = 2. Important progresses were made by Fan and Li [3], who advocated the unbiasedness and continuity as essential for variable selectors and carefully developed the SCAD method. In the theoretical front, Fan and Peng [4] proved that the SCAD has the oracle property when the number of variables is a certain fraction power of the sample size. However, nonconcave penalized likelihood methods are still commonly viewed as computationally limited and poorly understood, especially when the number of variables exceeds the number of data points. Zou and Li made two significant contributions by addressing both the computational and efficiency issues. They developed a fast iterative algorithm for minimizing nonconcave penalized likelihood and proposed a simple one-step method with the oracle property for full rank designs. We congratulate them on this important work. In what follows we relate our work to theirs through discussions on continuity, computational strategies, selection consistency and oracle efficiency.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/030e5d73c69926363a45ded611b3c7c085c7e645

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “Marginal regression xjy/‖xj‖2 can be used as an initial estimate of βj and is proved to result in the selection consistency of the adaptive LASSO under a certain partial orthogonality condition on the pairwise correlations among vectors {y,x1, . . . ,xp} [Huang, Ma and Zhang (2008)].”
VARIABLE SELECTION IN NONPARAMETRIC ADDITIVE MODELS (2010, methodology): “Then minimizing (2) subject to (3) is equivalent to minimizing (5) with respect to βn, but the centering constraints are not needed for (5).”
Least angle and ℓ1 penalized regression: A review (2008, background): “The nonnegative garrote is for example also studied in Gao (1998) and Bühlmann and Yu (2006). More recently, Yuan and Lin (2007b) proved some consistency results and showed that the solution path is piecewise linear and hence can be computed efficiently. The above methods try to reduce the bias of t”