SparseNet: Coordinate Descent With Nonconvex Penalties

By Rahul Mazumder, Jerome H. Friedman, Trevor J. Hastie. 2011

520 citations recorded by the source corpus.

Subfields: Computational Mechanics, Numerical Analysis, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Numerical Analysis

In Statistics and Probability

Citations in the corpus

Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso (2017, neutral): “, 1994); nonconvex-penalized methods, such as SCAD (Fan and Li, 2001), MC+ (Zhang, 2010), and SparseNet (Mazumder et al., 2011); hybrid lasso/stepwise approaches like FLASH (Radchenko and James, 2011); and many others.”
Spectral Regularization Algorithms for Learning Large Incomplete Matrices (2010, methodology): “We discuss simulations and experimental studies in Section 7 and application to the Netflix data in Section 8.”
Coordinate descent algorithms (2015, methodology): “The SparseNet algorithm [33] applied to problems with these same nonconvex separable regularizers uses warm-started cyclic coordinate descent as an inner loop to solve a sequence of problems in which the regularization parameter λ in (2) and the parameters deﬁning concavity of the regularization…”
A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding (2013, background): “Sparse coding [7, 18, 31] is an e ﬀ ective tool in a myriad of applications such as compressed sensing [11], image restoration [24, 25], face recognition [38], etc. Originally, it aims to solve the following minimization problem: where y is an n × 1 vector, A is an n × m redundant matrix with m > n…”
The Discrete Dantzig Selector: Estimating Sparse Linear Models via Mixed Integer Linear Optimization (2015, methodology): “In addition, from a statistical viewpoint, in terms of estimating sparse regression models subject to good data-fidelity, the Discrete Dantzig Selector may be perceived as a natural, interpretable and useful alternative to least squares with variable selection, Problem (4) — in the same way as the `”
The Spike-and-Slab LASSO (2018, methodology): “…LQA (Fan and Li 2001), the local linear approximation LLA (Zou and Li 2008; Candes, Wakin, and Boyd 2008), coordinate-wise optimization (Mazumder, Friedman, and Hastie 2011; Breheny and Huang 2011), proximal gradient methods (Loh and Wainwright 2014; Wang, Liu, and Zhang 2014), and…”
Degrees of freedom and model search (2014, methodology): “nd the rst sum above is exactly E Xn i=1 1fz it ig+ 1fz i t ig = EjA tj; as desired. Remark. This result, on the degrees of freedom of the hard thresholding operator, can be found in both Mazumder et al. (2011) and Deledalle et al. (2013). The former work uses degrees of freedom as a calibration”
STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION (2012, methodology): “Recently, coordinate descent was applied to solve the folded concave penalized least squares (Mazumder et al., 2011; Fan and Lv, 2011).”
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems (2011, methodology): “els supp(b) such as those generated in stepwise regression. For smooth concave penalties, iterative algorithms can be used to ﬁnd local minima of the penalized loss in (2) for a set of penalty levels [19, 50, 45, 4, 25, 47]. For the MCP and other quadratic spline concave penalties, a path following ”
Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation (2011, background): “In order to go beyond the ‘1-penalty, nonconvex penalties have been explored in recent literature [52], [54].”
An Iterative Coordinate Descent Algorithm for High-Dimensional Nonconvex Penalized Quantile Regression (2015, methodology): “For nonconvex penalized least squares regression, coordinate descent algorithms and their convergence theory were recently investigated by Breheny and Huang (2011) and Mazumder, Friedman, and Hastie (2011).”
Regularized M-estimators with nonconvexity: statistical and algorithmic theory for local optima (2013, methodology): “Various methods previously proposed for nonconvex optimization include local quadratic approximation (LQA) (Fan and Li, 2001), minorization-maximization (MM) (Hunter and Li, 2005), local linear approximation (LLA) (Zou and Li, 2008), and coordinate descent (Breheny and Huang, 2011; Mazumder et al., ”