Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression

By Arun K. Kuchibhotla, Abhishek Chakrabortty. 2018

133 citations recorded by the source corpus.

Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of independent random variables under much weaker exponential type (namely sub-Weibull) tail assumptions. These results extract a part sub-Gaussian tail behavior in finite samples, matching the asymptotics governed by the central limit theorem, and are compactly represented in terms of a new Orlicz quasi-norm - the Generalized Bernstein-Orlicz norm - that typifies such tail behaviors. We illustrate the usefulness of these inequalities through the analysis of four fundamental problems in HD statistics. In the first two problems, we study the rate of convergence of the sample covariance matrix in terms of the maximum elementwise norm and the maximum k-sub-matrix operator norm which are key quantities of interest in bootstrap, HD covariance matrix estimation and HD inference. The third example concerns the restricted eigenvalue condition, required in HD linear regression, which we verify for all sub-Weibull random vectors through a unified analysis, and also prove a more general result related to restricted strong convexity in the process. In the final example, we consider the Lasso estimator for linear regression and establish its rate of convergence under much weaker than usual tail assumptions (on the errors as well as the covariates), while also allowing for misspecified models and both fixed and random design. To our knowledge, these are the first such results for Lasso obtained in this generality. The common feature in all our results over all the examples is that the convergence rates under most exponential tails match the usual ones under sub-Gaussian assumptions.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/9045a9a223ec46da1bf7faeb2e2f0faccc9b4cd8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

High-dimensional central limit theorems by Stein’s method (2020, methodology): “Applying Theorem 3.1 of Kuchibhotla and Chakrabortty (2018) to the sequence of independent mean zero sub-Weibull (12) random variable (cf. (2.13)) {τjk(Xi) − jk : i = 1, . . . , n}, we have ∥∥∥∥∥1n n∑ i=1 ( τjk(Xi) − jk) ∥∥∥∥∥ 1 2 ,Ln ≤ C√ n , for some Ln = C/√n. Combining with Remark A.1 of Kuchibh”
Valid post-selection inference in model-free linear regression (2020, methodology): “The following theorem proved in Section 6 of Kuchibhotla and Chakrabortty (2018) provides a central limit theorem for SWn over all rectangles.”
Sub‐Weibull distributions: Generalizing sub‐Gaussian and sub‐Exponential properties to heavier tailed distributions (2019, methodology): “In addition, we illustrate their link with deep neural networks, not in the form of a model assumption as in Kuchibhotla and Chakrabortty (2018), but as a characterisation of the prior distribution of deep neural networks units.”