Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors

By Victor Chernozhukov, D. Chetverikov, Kengo Kato. 2012

571 citations recorded by the source corpus.

We derive a Gaussian approximation result for the maximum of a sum of high dimensional random vectors. Specifically, we establish conditions under which the distribution of the maximum is approximated by that of the maximum of a sum of the Gaussian random vectors with the same covariance matrices as the original vectors. This result applies when the dimension of random vectors (p) is large compared to the sample size (n); in fact, p can be much larger than n, without restricting correlations of the coordinates of these vectors. We also show that the distribution of the maximum of a sum of the random vectors with unknown covariance matrices can be consistently estimated by the distribution of the maximum of a sum of the conditional Gaussian random vectors obtained by multiplying the original vectors with i.i.d. Gaussian multipliers. This is the Gaussian multiplier (or wild) bootstrap procedure. Here too, p can be large or even much larger than n. These distributional approximations, either Gaussian or conditional Gaussian, yield a high-quality approximation to the distribution of the original maximum, often with approximation error decreasing polynomially in the sample size, and hence are of interest in many applications. We demonstrate how our Gaussian approximations and the multiplier bootstrap can be used for modern high dimensional estimation, multiple hypothesis testing, and adaptive specification testing. All these results contain non-asymptotic bounds on approximation errors.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

The hardness of conditional independence testing and the generalised covariance measure (2018, background): “[15] gives conditions under which Ĝ can well approximate the quantile function of a version of Sn where all bias terms, that is, terms corresponding to b, νg and νf are all equal to 0.”
Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, methodology): “Second, Chernozhukov et al. (2013) discuss finer data-driven choices of penalty levels based on the Gaussian or empirical bootstraps: λ = c× [(1− α)− quantile(‖D̂−1(M̂∗ + Ĝ∗t)‖∞ | (Wi)i∈Ic k )], where M̂∗ and Ĝ∗ are bootstrap copies of M̂ and Ĝ.”
Simultaneous Inference for High-Dimensional Linear Models (2016, methodology): “Let Chernozhukov et al. (2013) and the proof of Theorem 2.2, we can establish the asymptotic equivalence between the empirical bootstrap and the multiplier bootstrap, which theoretically justiﬁes the use of the former.”
A multivariate Berry–Esseen theorem with explicit constants (2018, methodology): “We recall the procedure briefly; for an exposition, see Röllin [21] and Appendix H of Chernozhukov, Chetverikov and Kato [12].”
Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications (2016, methodology): “The Gaussian and bootstrap approximations for the maxima of sums of high-dimensional independent random vectors were considered in [24, 22].”
Targeted Undersmoothing. (2017, methodology): “In addition, [25], [24] and [29] describe how bootstrapping can be used in conjunction with some of the previously cited techniques.”
Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores (2017, unknown): “Moreover, in the context of linear regression with p n Bradic (2013), Chatterjee and Lahiri (2013), Liu and Yu (2013), Chernozhukov et al. (2013) and Lopes (2014) study properties of various bootstrap methods.”
Orthogonal Machine Learning for Demand Estimation: High Dimensional Causal Inference in Dynamic Panels (2017, methodology): “l0(Zit) In other words, there is no unobserved unit heterogeneity .Then, small bias condition is achieved by many ML methods. Assumption 3.2 holds under plausible β-mixing conditions on Zit (see e.g. Chernozhukov et al. (2013b).) Remark 4.1 (Partialling out individual heterogeneity). Partialling out”
Bootstrapping and sample splitting for high-dimensional, assumption-lean inference (2016, methodology): “In both cases, the Berry-Esseen theorem for simple convex sets (polyhedra with a limited number of faces) with increasing dimension due to Chernozhukov et al. (2013, 2014) justifies the method.”
Nearly optimal central limit theorem and bootstrap approximations in high dimensions (2020, methodology): “l in mathematical statistics, and, at the same time, it is small enough so that, as n→ ∞, under minimal conditions, we have = n→ 0 even if d= dn→ ∞ much faster than n→ ∞, as was originally shown in [12], making bounds on particularly useful in high-dimensional statistics and machine learning, e.g. i”
Frequentist coverage and sup-norm convergence rate in Gaussian process regression (2017, methodology): “Our construction is closely related to the construction in [13] (they consider approximating the indicator function 1 A ( x ) for any measurable subset A , therefore their construction is more complicated), and using their Lemma 4.3 we obtain Step 3: We need the following multivariate version of…”
High-dimensional central limit theorems by Stein’s method (2020, unknown): “3 with a truncation argument, we can improve Chernozhukov et al. ((2019), Theorem 2.”
Bootstrap Methods in Econometrics (2018, unknown): “Chernozhukov, Chetverikov, and Kato (2013) (CCK) consider the problem of estimating the probability distribution of ( ) 1 max j j p Z X ≤ ≤ = when p may be greater than n .”