On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators

By Noureddine El Karoui. 2018

140 citations recorded by the source corpus.

We study ridge-regularized generalized robust regression estimators, i.e. $$\begin{aligned} {\widehat{\beta }}={{\mathrm{argmin}}}_{\beta \in {\mathbb {R}}^p} \frac{1}{n}\sum _{i=1}^n \rho _i(Y_i-X_i'\beta )+\frac{\tau }{2}||\beta ||^2, \quad \text {where } \quad Y_i=\epsilon _i+X_i'\beta _0, \end{aligned}$$ in the situation where p/n tends to a finite non-zero limit. Our study here focuses on the situation where the errors \(\epsilon _i\)’s are heavy-tailed and \(X_i\)’s have an “elliptical-like” distribution. Our assumptions are quite general and we do not require homoskedasticity of \(\epsilon _i\)’s for instance. We obtain a characterization of the limit of \(||{\widehat{\beta }}-\beta _0||\), as well as several other results, including central limit theorems for the entries of \({\widehat{\beta }}\).

Subfields: Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b2f8099d748cec0231919c6214cb99db074bb2f6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yuxin Chen
2. Yuejie Chi
3. Cong Ma
4. Jaouad Mourtada
5. Kaizheng Wang
6. Navid Azizan
7. Babak Hassibi

In Statistics and Probability

1. Yuejie Chi
2. Cong Ma
3. Kaizheng Wang
4. Jaouad Mourtada
5. Yuxin Chen

Citations in the corpus

• Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, background): “For instance, [26] obtains different limiting risks for robust regression in high dimension when considering non-independent coordinates.”
• Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution (2017, methodology): “In particular, this technique results in tight performance guarantees for the generalized power method [133], the spectral method [1,30], and convex programming approaches [30,44,108,133]; however, it has not been applied to analyze nonconvex optimization algorithms.”
• Spectral Method and Regularized MLE Are Both Optimal for Top-$K$ Ranking (2017, unknown): “…noting that the leave-one-out idea has been invoked to analyze random designs for other high-dimensional problems as well, e.g. robust M-estimators (El Karoui et al., 2013; El Karoui, 2017), confidence intervals for Lasso (Javanmard and Montanari, 2015), likelihood ratio test (Sur et al., 2017).”