High-Dimensional Statistics

By Martin J. Wainwright. 2019

680 citations recorded by the source corpus.

Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.

Subfields: Artificial Intelligence, Computational Mechanics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

Citations in the corpus

Additive models with trend filtering (2017, background): “2 in Wainwright [2017], which is a somewhat unique theorem, because it does not require a global sup norm bound on the function class in consideration (unlike many standard results of this type); instead, the theorem requires componentwise sup norm bounds (and even this can be relaxed).”
The Augmented Synthetic Control Method (2018, methodology): “, 2017) and the generalized synthetic control method (Xu, 2017), and off-the-shelf machine learning models like random forests and neural networks. In addition, ASCM can accommodate auxiliary, time-invariant covariates to further reduce bias. Despite these advantages, we recommend that, as with any ”
Adversarially Robust Generalization Just Requires More Unlabeled Data (2019, methodology): “The following theorem from Wainwright (2019) characterizes the concentration property of spiked covariance ensemble, which we will further use to bound the robust classification error.”
Estimating Sparse Discrete Distributions Under Local Privacy and Communication Constraints (2020, background): “genomics, and language modeling, the distribution is often supported over a small unknown subset of the domain. Motivated by the utility gain in high dimensional statistics under sparsity assumptions [33], we study the problem of estimating sparse discrete distributions under privacy and communicati”
Learning Two Layer Rectified Neural Networks in Polynomial Time (2018, neutral): “otherwise). Moreover, the mean of the entries of f(ϕ(XS)k,∗) is Θ(ϕ). The L2 of these vectors are thus sums of sub-exponential random variables, so by standard sub-exponential concentration (see e.g. [Wai19]) we have kf(ϕ(XS)k,∗)(I−P1)k2 = Θ(ϕ) p |S|,and moreover kf(ϕ(XS)k,∗)k2 −kf(ϕ(XS)k,∗)(I−P1)k2”
CoinPress: Practical Private Mean and Covariance Estimation (2020, neutral): “Lemma C.3 ((6.12) of [Wai19]) .”
Statistically Efficient Off-Policy Policy Gradients (2020, background): “r d^ j . Examples for Finclude RKHS norm balls, an expanding subspace of L 2 (i.e., a sieve), and neural networks. Rates for such estimators can, for example, be derived from Bartlett et al. (2005); Wainwright (2019). A careful reader might wonder whether estimating nuisances in this way causes the ”
Random graph asymptotics for treatment effect estimation under network interference (2020, neutral): “…random variable X with mean µ= E[X] and variance σ2 = E [ X2 ] − µ2, we say that Bernstein’s condition with parameter b holds if (39) ∣∣∣E[(X − µ)k]∣∣∣≤ 1 2 k!σ2bk−2 for k = 2,3,4, . . . . Wainwright (2019) shows that one sufficient condition for Bernstein’s condition to hold is that X be bounded.”
An Efficient Framework for Clustered Federated Learning (2020, methodology): “According to the minimax estimation rate of linear regression [40], we know that even if we know the ground truth cluster identities, we cannot obtain an error rate better than O(σ √ d pmn′ log(∆/4ε) ).”
Fairness with Dynamics (2019, neutral): “See (Wainwright, 2019) for a proof.”
Uniform convergence may be unable to explain generalization in deep learning (2019, methodology): “First, we state a tail bound for sub-exponential random variables (Wainwright, 2019).”
High‐dimensional quantile regression: Convolution smoothing and concave regularization (2021, background): “Because of the extremely long list of relevant literature, we refer the reader to the monographs Bühlmann and van de Geer (2011), Hastie et al. (2015), Wainwright (2019), Fan et al. (2020), and the references therein.”
Human Interaction with Recommendation Systems (2017, neutral): “(27) Bounding Am Using the standard σ-sub-Gaussian concentration bound (see, for example, [Wainwright, 2015, Chapter 2]), we have P(Am(i, λ)) ≤ P ( ∃s ∈ {m, . . . , t} : 1 s ∣∣∣∣∣ s∑ i=1 εi ∣∣∣∣∣ > λ ) (28) ≤ t∑ s=m P ( 1 s ∣∣∣∣∣ s∑ i=1 εi ∣∣∣∣∣ > λ ) (29) ≤ 2 t∑ s=m exp ( − sλ 2 2σ2 ) (30) ≤ 2 ∫…”
On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression (2020, background): “ N XN i=1 σih(xi) # . (4) where σ1,σ2,...,σN are the Rademacher random variables, which are deﬁned by the uniform distribution as: σi = ( 1, with probability 1 2 -1, with probability 1 2. (5) In [32]–[34], it was shown that a function class with larger empirical Rademacher complexity is more likely ”