Misspecification in infinite-dimensional Bayesian statistics

By B. Kleijn, W. A., Van der Vaart. 2006

232 citations recorded by the source corpus.

We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution P 0 , which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback-Leibler divergence with respect to P 0 . An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/a18f95f80df905e2b3c9f1d28cbfdc2f2fc2562e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Posterior convergence rates of Dirichlet mixtures at smooth densities (2007, background): “The proof follows by a minor adaptation of a result in [11], pages 475–479, as in [10].”
On posterior contraction of parameters and interpretability in Bayesian mixture modeling (2019, methodology): “Due to the lack of convexity of the class of mixture densities with bounded number of mixture components, the theory developed in this section (tracing back to the work of [27]) is not applicable.”
Suboptimal behavior of Bayes and MDL in classification under misspecification (2004, background): “for which Bayes is inconsistent with D-probability 1. This is interesting because even under misspecification, Bayes is consistent under fairly broad conditions (Bunke & Milhaud, 1998; Kleijn & van der Vaart, 2004 ), in the sense that the posterior concentrates on a neighborhood of the distribution”
The Bernstein-Von-Mises theorem under misspecification (2012, methodology): “1 in Kleijn and Van der Vaart (2003) [9].”
Bayesian fractional posteriors (2016, background): “As one of our contributions, we show that the contraction rate of the fractional posterior is entirely determined by the prior mass assigned to appropriate KL neighborhoods of the true distribution, bypassing the construction of sieves and testing arguments in the existing theory [4,23,32].”
Dynamics of Bayesian Updating with Dependent Data and Misspecified Models (2009, methodology): “[38, 68, 5, 6] on mis-specified models do not hold because the data are dependent.”
The semiparametric Bernstein-von Mises theorem (2010, background): “For posterior concentration to occur [16, 27] sufficient prior mass must be present in certain Kullback–Leibler-type neigh-”
Critical dimension in the semiparametric Bernstein—von Mises theorem (2014, background): “Under model misspecification, υ∗ defines the best parametric fit to P by the considered family (see [11, 19, 20] and references therein).”
The Safe Bayesian - Learning the Learning Rate via the Mixability Gap (2012, background): “This lemma slightly extends a result by Barron and Li (1999). It is proved as Proposition 5 in Grünwald (2011) (in different context, but the modification to our setting is immediate).”
Bernstein - von Mises Theorem for growing parameter dimension (2013, background): “Only few results are available which admit some violation from the parametric assumption; see e.g. Kleijn and van der Vaart (2006), Kleijn and van der Vaart (2012).”