Analysis of Multi-stage Convex Relaxation for Sparse Regularization

By Tong Zhang. 2010

583 citations recorded by the source corpus.

Subfields: Civil and Structural Engineering, Computational Mechanics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Civil and Structural Engineering

In Computational Mechanics

In Control and Systems Engineering

Citations in the corpus

Best Subset Selection via a Modern Optimization Lens (2015, methodology): “…used to “bridge” the gap between the convex 1 penalty and the combinatorial 0 penalty [Candès, Wakin and Boyd (2008), Fan and Li (2001), Frank and Friedman (1993), Friedman (2008), Mazumder, Friedman and Hastie (2011), Zhang (2010a, 2010b), Zhang and Huang (2008), Zou (2006), Zou and Li (2008)].”
Multi-Stage Multi-Task Feature Learning (2012, positive): “Different from previous work on multi-stage sparse learning which focuses on a single task (Zhang, 2010, 2012), we study a more general multi-stage framework in the multi-task setting.”
Iterative reweighted algorithms for matrix rank minimization (2012, unknown): “It is shown theoretically that for sparse recovery from noisy measurements, this algorithm has a better recovery error than ℓ 1 minimization under suitable assumptions on the matrix A (Needell, 2009; Zhang, 2010).”
Generalized Nonconvex Nonsmooth Low-Rank Minimization (2014, methodology): “Capped L1 [24] { λθ, if θ < γ, λγ, if θ ≥ γ.”
A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems (2013, methodology): “Although the ℓ 1 - norm based sparse learning formulations have achieved great success, they have been shown to be suboptimal in many cases ( Candes et al., 2008; Zhang, 2010b; 2012), since the ℓ 1 -norm is a loose approximation of the ℓ 0 -norm and often leads to an over-penalized problem.”
An Exact Penalty Method for Binary Optimization Based on MPEC Formulation (2017, methodology): “For example, the work of (Zhang, 2010) considers a multi-stage convex optimization algorithm to refine the global solution by the initial convex method; the work of (Candès, Li, and Soltanolkotabi, 2015) starts with a careful initialization obtained by a spectral method and improves this estimate…”
Capped Lp-Norm Graph Embedding for Photo Clustering (2016, methodology): “For example, capped (cid:96) 1 -norm is used for sparsity regularization [23].”
Multi-Stage Multi-Task Learning with Reduced Rank (2016, background): “Recently, the capped norms (Zhang 2009; 2010; 2011; Gong, Ye, and Zhang 2012; Sun, Xiang, and Ye 2013) have gained the popularity since those norms are capable of penalizing part of the parameters lower than a threshold.”
Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method (2021, background): “To address this, researchers have explored nonconvex surrogates for the ℓ 0 norm, such as Cappled L 1 [51], Smoothly Clipped Absolute Deviation (SCAD)[18] and Minimax Concave Penalty (MCP)[48], as alternative solutions, aiming to develop more effective and efficient algorithms.”
Efficient Methods for Overlapping Group Lasso (2011, methodology): “Several recent works use the nonconvex capped norm that is closer to the ‘0 norm than the ‘1 norm as follows [29], [30], [31]: kxk0 Xp”
Enhancing one-class support vector machines for unsupervised anomaly detection (2013, methodology): “[32] Wenjian Wang, Zongben Xu, Weizhen Lu, and Xiaoyun Zhang.”
What Objective Does Self-paced Learning Indeed Optimize? (2015, background): “, the hard and linear SPL regimes are equivalent to the optimizations on losses with the capped-norm penalty [11, 12, 13] and minimax concave plus penalty (MCP) [14], respectively.”
Accelerated Proximal Gradient Methods for Nonconvex Programming (2015, methodology): “To address this issue, many nonconvex regularizers have been proposed, such as lp-norm [4], Capped-l1 penalty [3], Log-Sum Penalty [2], Minimax Concave Penalty [5], Geman Penalty [6], Smoothly Clipped Absolute Deviation [7] and Schatten-p norm [8].”
High‐dimensional quantile regression: Convolution smoothing and concave regularization (2021, methodology): “As discussed in Fan and Lv (2008), an advantage of the concave penalty, such as SCAD and MCP, is that zero is not an absorbing state: once a coefficient is shrunk to zero, it will remain zero throughout the remaining iterations.”
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems (2011, methodology): “Another simple concave penalty is the capped- 1 penalty introduced in [46].”
SparseNet: Coordinate Descent With Nonconvex Penalties (2011, methodology): “Zou and Li (2008), Zhang (2010b) and Candes, Wakin, and Boyd (2008) propose majorize-minimize (MM) algorithms for minimization of Q(fi) (6), and consider the reweighted l\ criterion Q(fi; Pk)= Constant + i ||y-Xj81|2 There is currently a lot of attention payed to MLLA in the lit erature, so we…”
STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION (2012, methodology): “Zhang (2010b, 2013) analyzed the capped-ℓ 1 penalty for solving the penalized least squares and proved the oracle property as well.”
A Selective Review of Group Selection in High-Dimensional Models. (2012, methodology): “This question was addressed by Huang and Zhang (2010), who introduced the concept of strong group sparsity.”
Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm (2015, methodology): “Capped L1 [17] { λθ, if θ < γ, λγ, if θ ≥ γ.”