Rates of contraction of posterior distributions based on Gaussian process priors

By W. A., Van der Vaart, H. J., V. Zanten. 2008

476 citations recorded by the source corpus.

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Density Estimation in Infinite Dimensional Exponential Families (2013, methodology): “The convergence rate for KL in Theorem 6 is based on the application of Theorem 5(ii) in Lemma A.1, where the bound on KL in Lemma A.1—quoted from [40, Lemma 3.1]—critically uses the boundedness to upper bound KL in terms of squared Hellinger distance.”
On Bayesian supremum norm contraction rates (2013, methodology): “Ghosal, Ghosh and van der Vaart [12], Scricciolo [28], Tokdar and Ghosh [34], van der Vaart and van Zanten [37], [3], Rivoirard and Rousseau [26], and random dyadic histograms, for random histogram priors see e.”
Bayesian inverse problems with non-conjugate priors (2012, background): “The mildly ill-posed case is dealt with in [20] using the conjugacy of the prior and we recover the same rates using our testing approach combined with the results of [31] (though we do not consider the case of scaling).”
Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs (2012, unknown): “Since the regularity of the prior we employ in this paper is essentially α = p − 1 / 2, this suggests that in principle, it should be possible to relax our assumption that b 0 is p -regular to the assumption that b 0 is ( p − 1 / 2 ) -regular, while still maintaining the same rate T − ( p − 1 / 2…”
Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems (2012, methodology): “Our methods assume a relatively weak relation between the prior covariance, noise covariance and forward operator, allowing for a wide range of applications. c⃝ 2013 Elsevier B.V.”
Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions (2018, background): “robability approaching one. We now turn to the proof of Theorem 1 and verify the conditions (17)-(20) of Theorem 7. By Proposition 1, k · kL2 and k · kµ0 are equivalent norms. Applying Theorem 4.5 of [53] (see also Sections 11.3 and 11.4.5 in [21]), there exist measurable sets BT ⊂ VJ such that for ”
Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator (2015, background): “In van der Vaart and van Zanten [31], the prior had some natural Banach structure and”
Information Rates of Nonparametric Gaussian Process Methods (2011, methodology): “This is in accordance with the findings for other GP priors in in Van der Vaart and Van Zanten (2008a).”
Nonparametric Bernstein–von Mises theorems in Gaussian white noise (2012, background): “Although remarkable progress has been made in the understanding of the frequentist properties of nonparametric Bayes procedures (we refer here only to some of the key papers such as [18, 19, 34, 36, 38] and references therein), a fully satisfactory answer to the BvM-question seems not to have been f”
On adaptive inference and confidence bands (2011, methodology): “Gaussians, which leads to measures that have a structure similar to Gaussian priors used in Bayesian nonparametrics; see, for example, [30].”
Non-Gaussian statistical inverse problems. Part I: Posterior distributions (2012, background): “The case of Gaussian regression has been studied in [131], where certain probabilities are shown to converge.”
Nonparametric Bayesian posterior contraction rates for discretely observed scalar diffusions (2015, background): “[9], see also [10] and [33] – but the adaptation to the diffusion case requires the resolution of two major mathematical obstacles to obtain satisfactory results: • The ‘small ball probability conditions’ need to reflect the inverse problem nature of the discrete sampling scheme, and the resulting p”
Limits of Estimating Heterogeneous Treatment Effects: Guidelines for Practical Algorithm Design (2018, background): “Potential choices for the basis (cid:22) φ (cid:12) w that would give rise to implementable Bayesian inference algorithms include regular wavelet basis (Zhang, 1997), radial basis for a reproducing kernel Hilbert space (RKHS) (van der Vaart et al., 2008), etc.”
On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures (2013, unknown): “The resulting posterior distributions (·|X(n)) attain minimax optimal contraction rates up to logarithmic terms in Hellinger and L2-distance [5, 32, 37] and L∞-distance [3].”
Reproducing kernel Hilbert spaces of Gaussian priors (2008, methodology): “It also covers logistic or probit regression as considered in [5] with appropriate choices and several other situations, as shown in [15].”