Asymptotic properties of bridge estimators in sparse high-dimensional regression models

By Jian Huang, Joel L. Horowitz, Shuangge Ma. 2008

527 citations recorded by the source corpus.

We study the asymptotic properties of bridge estimators in sparse, high-dimensional, linear regression models when the number of covariates may increase to infinity with the sample size. We are particularly interested in the use of bridge estimators to distinguish between covariates whose coefficients are zero and covariates whose coefficients are nonzero. We show that under appropriate conditions, bridge estimators correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance. Thus, bridge estimators have an oracle property in the sense of Fan and Li [J. Amer. Statist. Assoc. 96 (2001) 1348-1360] and Fan and Peng [Ann. Statist. 32 (2004) 928-961]. In general, the oracle property holds only if the number of covariates is smaller than the sample size. However, under a partial orthogonality condition in which the covariates of the zero coefficients are uncorrelated or weakly correlated with the covariates of nonzero coefficients, we show that marginal bridge estimators can correctly distinguish between covariates with nonzero and zero coefficients with probability converging to one even when the number of covariates is greater than the sample size.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e28ba0fd6ea94bd56f277d3a89de8f47fd5d1943

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Forward Regression for Ultra-High Dimensional Variable Screening (2009, methodology): “All those methods are very useful and can be formulated as penalized optimization problems, which could be selection consistent, if the sample size is much larger than the predictor dimension (Fan and Peng, 2004; Huang et al., 2008; Zou and Zhang, 2008).”
Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, unknown): “The bridge estimator [45] is defined as follows:”
Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming (2010, result): “entity T of signiﬁcant regressors, and so it can achieve the rate of σ p s/n.) The result above was demonstrated by [2], and closely related results were given in [7], [19], [4], and [16]. [3], [20], [5], [6], and [18] contain other fundamental results obtained for related problems; please see [2] f”
Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models (2009, methodology): “ al. (2009) used a diﬀerent marginal utility, derived from an empirical likelihood point of view. Hall and Miller (2009) proposed a generalized correlation ranking, which allows nonlinear regression. Huang et al. (2008) also investigated the marginal bridge regression in the ordinary linear model. T”
Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap (2013, methodology): “…and B¨uhlmann (2006), Wainwright (2006), Bunea, Tsybakov and Wegkamp (2007), Bickel, Ritov and Tsybakov (2009), Huang, Ma and Zhang (2008), Huang, Horowitz and Ma (2008), Zhang and Huang (2008), Meinshausen and Yu (2009), P¨otscher and Schneider (2009), Chatterjee and Lahiri (2011b),…”
Adaptive Lasso for sparse high-dimensional regression models (2008, methodology): “This theorem can be proved by verifying the Lindeberg conditions in the same way as in the proof of Theorem 2 of Huang et al. (2008). Thus we omit the proof here.”
Shrinkage Estimation of Dynamic Panel Data Models with Interactive Fixed Effects (2016, unknown): “Zou (2006) establishes the oracle property of adaptive Lasso; Yuan and Lin (2006) propose the method of group Lasso; Wang and Leng (2008) and Wei and Huang (2010) study the properties of adaptive group Lasso; Huang et al. (2008) study Bridge estimators in sparse high dimensional regression models.”
Sure independence screening for ultrahigh dimensional feature space (2006, background): “which along with (37) and Condition 1 shows that”
Asymptotically Honest Confidence Regions for High Dimensional Parameters by the Desparsified Conservative Lasso (2014, background): “uch greater than the sample size. At ﬁrst, much focus was devoted to establishing the so-called oracle property in models of ﬁxed or increasing dimensions, see e.g. Fan and Li (2001), Zou (2006), and Huang et al. (2008). This entails showing that the procedure asymptotically detects the correct, and”
Bootstrapping Lasso Estimators (2011, background): “In the high-dimensional case, where p is allowed to grow with n, work on estimation consistency of the Lasso is limited; among them Huang, Horowitz, and Ma (2008) considered the asymptotic properties of q norm penalized regression estimators (0 < q < 1) for high-dimensional regression models.”
Variable selection for high-dimensional generalized varying-coefficient models (2012, background): “…has been made on the problem of variable selection for linear models and generalized linear models (Tibshirani (1996); Fan and Li (2001); Fan and Peng (2004); Zou (2006); Zou and Li (2008); Yuan and Lin (2007); Huang, Horowitz, and Ma (2008); Choi, Li, and Zhu (2010); Li, Peng, and Zhu (2011)).”
A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, methodology): “More recent studies on this topic include Huang, Horowitz and Ma (2008), Kim, Choi and Oh (2008), Meier, van de Geer and B¨uhlmann (2008), Lv and Fan (2009), Zhang (2009), and Fan and Lv (2009), among others.”
Smoothing methods for nonsmooth, nonconvex minimization (2012, background): “An increasing number of practical problems require minimizing a nonsmooth, nonconvex function on a convex set, including image restoration, signal reconstruction, variable selection, optimal control, stochastic equilibrium problems and spherical approximations [1,8,10,11,13,21,29,30,33,42,44, 61,76,”
Non-Concave Penalized Likelihood with NP-Dimensionality (2009, methodology): “In this section we discuss the choice of penalty functions in regularization methods and characterize the non-concave penalized likelihood estimator as well as its global optimality.”
Shrinkage tuning parameter selection with a diverging number of parameters (2009, background): “This assumption can be relaxed but at the cost of more complicated technical proofs and certain assumptions for ε’s tail heaviness; see Huang et al. (2007b). Theorem 1.”
Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation (2007, methodology): “Following a similar argument to Huang, Horowitz and Ma (2008), the local minimizer in Theorem 1 can be taken as the global minimizer with additional conditions on the tail of the penalty function.”