Gaussian approximation of suprema of empirical processes

By Victor Chernozhukov, D. Chetverikov, Kengo Kato. 2012

329 citations recorded by the source corpus.

We develop a new direct approach to approximating suprema of general empirical processes by a sequence of suprema of Gaussian processes, without taking the route of approximating empirical processes themselves in the sup-norm. We prove an abstract approximation theorem that is applicable to a wide variety of problems, primarily in statistics. Especially, the bound in the main approximation theorem is non-asymptotic and the theorem does not require uniform boundedness of the class of functions. The proof of the approximation theorem builds on a new coupling inequality for maxima of sums of random vectors, the proof of which depends on an effective use of Stein's method for normal approximation, and some new empirical processes techniques. We study applications of this approximation theorem to local empirical processes and series estimation in nonparametric regression where the classes of functions change with the sample size and are not Donsker-type. Importantly, our new technique is able to prove the Gaussian approximation for the supremum type statistics under considerably weak regularity conditions, especially concerning the bandwidth and the number of series functions, in those examples.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/25d51b3d9e57e888759ef5e5698fc8e91e4580f5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Empirical process results for exchangeable arrays (2019, methodology): “Finiteness of the uniform entropy integral is satisfied by any VC-type class of functions (see Chernozhukov et al., 2014, for a definition), or by the convex hull of such classes under some restrictions.”
Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression (2018, background): “) 1jp 1 n Xn i=1 Z i ; where X i2Rpare mean zero and Z i˘N p(0;E X iX> i ) constructed on the same probability space as X i’s. For this quantity to converge in probability to zero, Theorem 4.1 ofChernozhukov et al. (2014) requires nto converge to zero, among other terms. 4.1.1 Gram Matrix to Cova”
Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors (2012, methodology): “Lastly, in a companion work ([10]), we derive the strong coupling for suprema of general empirical processes based on the methods developed here and maximal inequalities.”
High-dimensional econometrics and regularized GMM (2018, background): “are given in Lemmas 8 and 9 of [49]. The proof of the last inequality can be found in [24]. 94 HD ECONOMETRICS AND REGULARIZED GMM The last result follows from the second inequality and Theorem 5.1 in[47] with = 1. Indeed we have with probability 1 tq= 2 max j2[p] jG n(X j)j2 n m 2 p logp+ n 1=2M 2 ”
Convergence rates of least squares regression estimators with heavy-tailed errors (2017, unknown): “by Fβ(x) = β−1log Pd i=1exp(βxi) . Then supx∈Rd Pd j,k,l,m=1|∂jklmFβ(x)| ≤ 25β3. Proof. Let πj(x) = ∂jFβ(x) = exp(βxj)/ Pd i=1exp(βxi) and δij = 1i=j. Then it is easy to verify that (see Lemma 4.3 of [22]) ∂jkFβ = ∂j(πk) = β(δjkπj −πjπk), ∂jklFβ = β 2 δjkδjlπl −δjkπjπl +2πjπkπl −(δjl +δkl)πkπl . Fur”
Estimation and Inference about Heterogeneous Treatment Effects in High-Dimensional Dynamic Panels. (2017, methodology): “ity is also available in Kock and Tang (2019) for a simpler model than ours. In addition, we contribute to a growing literature on high-dimensional Gaussian approximation (Chernozhukov et al. (2013), Chernozhukov et al. (2014), Chernozhukov et al. (2015), Chernozhukov et al. (2017b), Zhang and Wu (2”
Some new asymptotic theory for least squares series: Pointwise and uniform results (2012, methodology): “:= ℓ(w)′Ωb1/2N k/ √ n bσθ(w) ,w∈ I and Nk ∼ N(0,Ik). Note that cn(1 − α) deﬁned in (5.40) can be approximated numerically by simulation. Yet, conditions of Theorem 5.4 are rather strong. Fortunately, Chernozhukov et al. (2012a) noticed that when we are only interested in the supremum 30 BELLONI,CHER”
Central limit theorems and bootstrap in high dimensions (2014, background): “In addition, examples in [19] suggest that dependence on Bn is also optimal.”
Empirical and multiplier bootstraps for suprema of empirical processes of increasing complexity, and related Gaussian couplings (2015, unknown): “This paper builds upon but differs from our previous papers [5,9,6,8,7].”
Double/Debiased Machine Learning for Treatment and Causal Parameters (2016, methodology): “alues in a measurable space (W;A W) according to a probability law P. Let Fbe a set of suitably measurable functions f: W!R, equipped with a measurable envelope F: W!R. Lemma A.1 (Maximal Inequality, [25]). Work with the setup above. Suppose that F>sup f2Fjfj is a measurable envelope for Fwith kF”
Frequentist coverage and sup-norm convergence rate in Gaussian process regression (2017, methodology): “By applying Theorem 2.1 in [15] about a Gaussian approximation to the suprema of empirical processes (similar calculations as Corollary 2.2 therein), there exists a random variable B n that has the same distribution as (cid:98) Z Mn = (cid:107) (cid:99) W (cid:107) ∞ , the suprema of the GP…”
Program evaluation with high-dimensional data (2013, background): “The prior approaches dealing with continuums of moment conditions with infinite-dimensional nuisance parameters, for example, the ones given in Chernozhukov and Hansen (2006) and Escanciano and Zhu (2013), impose the Donsker conditions on the class of functions, following Andrews (1994a), that conta”