Smoothly Clipped Absolute Deviation on High Dimensions

By Yongdai Kim, Hosik Choi, Hee‐Seok Oh. 2008

277 citations recorded by the source corpus.

The smoothly clipped absolute deviation (SCAD) estimator, proposed by Fan and Li, has many desirable properties, including continuity, sparsity, and unbiasedness. The SCAD estimator also has the (asymptotically) oracle property when the dimension of covariates is fixed or diverges more slowly than the sample size. In this article we study the SCAD estimator in high-dimensional settings where the dimension of covariates can be much larger than the sample size. First, we develop an efficient optimization algorithm that is fast and always converges to a local minimum. Second, we prove that the SCAD estimator still has the oracle property on high-dimensional problems. We perform numerical studies to compare the SCAD estimator with the LASSO and SIS–SCAD estimators in terms of prediction accuracy and variable selectivity when the true model is sparse. Through the simulation, we show that the variance estimator of Fan and Li still works well for some limited high-dimensional cases where the true nonzero coefficients are not too small and the sample size is moderately large. We apply the proposed algorithm to analyze a high-dimensional microarray data set.

Subfields: Computational Mechanics, Molecular Biology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d9b5e89b643aa852b5d9ecdaeef43d7b5bc13824

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Lan Wang
2. Cun-Hui Zhang
3. Shenmin Zhang
4. Jianqing Fan
5. Jinchi Lv
6. Tong Zhang
7. Wen-Xin Zhou
8. Kean Ming Tan

In Molecular Biology

1. Jinchi Lv
2. Cun-Hui Zhang
3. Xiaotong Shen
4. Wei Zhong
5. Runze Li
6. Liping Zhu
7. Shenmin Zhang
8. Tong Zhang

In Statistics and Probability

1. Cun-Hui Zhang
2. Jianqing Fan
3. Lan Wang
4. Peter Buhlmann
5. Jinchi Lv
6. Tong Zhang
7. Shenmin Zhang
8. Yingcun Xia

Citations in the corpus

• ADAPTIVE SEMI-VARYING COEFFICIENT MODEL SELECTION (2012, methodology): “Further research includes extending the proposed procedure to longitude data and to the case with a diverging number of parameters (Kim, Choi, and Oh (2008), Lam and Fan (2008)).”
• A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, background): “…eignvalue of the correlation matrix of those active variables is bounded away from zero, and the minimum signal min 1 j s n Under Condition A, Kim, Choi and Oh (2008) prove that if Eε 2 ki ∞ for the linear model (3), P (ˆ β o is a local minima of PLS with the SCAD penalty) → 1 , (46) This…”
• Partially functional linear regression in high dimensions (2016, methodology): “We modify the concave-convex procedure used in Kim et al. (2008).”
• High‐dimensional quantile regression: Convolution smoothing and concave regularization (2021, methodology): “(2006), Kim et al. (2008) and Wang et al. (2012). The dataset was collected on a study that used eQTL mapping in laboratory rats to investigate and identify genetic variation in the mammalian eye that is relevant to human eye disease (Scheetz et al., 2006) Following Wang et al. (2012), we study the ”
• Machine Learning Advances for Time Series Forecasting (2020, background): “Fan and Li (2001) show model selection consistency and oracle property for the folded concave penalty estimators in a fixed dimensional setting.”
• Feature Screening via Distance Correlation Learning (2012, methodology): “We develop a sure independence screening procedure based on the distance correlation (DC-SIS).”
• Penalized expectile regression: an alternative to penalized quantile regression (2018, methodology): “For the SCAD penalty with (5), because the objective function Rn(β; τ) in (4) is not convex, we apply the concave–convex procedure (CCCP) (Yuille and Rangarajan 2003; Zou and Li 2008; Kim et al. 2008).”
• Confidence intervals for low dimensional parameters in high dimensional linear models (2011, methodology): “…2010; Koltchinskii et al., 2011; Sun and Zhang, 2010), non-convex penalized methods (Frank and Friedman, 1993; Fan and Li, 2001; Fan and Peng, 2004; Kim et al., 2008; Zhang, 2010; Zhang and Zhang, 2012), greedy methods (Zhang, 2011a), adaptive methods (Zou, 2006; Huang et al., 2008; Zhang,…”
• A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems (2011, methodology): “Another relevant study is [20], which showed that in the lower-dimensional scenario with p ≤ n, the global solution of (2) agrees with the oracle estimator β̂ for the SCAD penalty”
• On constrained and regularized high-dimensional regression (2013, background): “For the smoothly clipped absolute deviation (SCAD; Fan and Li 2001) regularization, Kim et al. (2008) and Lv and Fan (2009) showed that some consistent local minimizers exist for SCAD.”