Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems

By Maxim Panov, Vladimir Spokoiny. 2013

46 citations recorded by the source corpus.

The classical parametric and semiparametric Bernstein -- von Mises (BvM) results are reconsidered in a non-classical setup allowing finite samples and model misspecification. In the case of a finite dimensional nuisance parameter we obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter which is explicit in the dimension of the nuisance and target parameters. This helps to identify the so called \emph{critical dimension} $ p $ of the full parameter for which the BvM result is applicable. In the important i.i.d. case, we show that the condition "$ p^{3} / n $ is small" is sufficient for BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension $ p $ approaches $ n^{1/3} $. The results are extended to the case of infinite dimensional parameters with the nuisance parameter from a Sobolev class. In particular we show near normality of the posterior if the smoothness parameter $s$ exceeds 3/2.

Subfields: Artificial Intelligence, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “It is worth noting that the Laplace approximation to the RFF posterior is asymptotically exact by the virtue of the Bernstein-von Mises (BvM) theorem and the fact that (3) is a finite-rank model (Dehaene, 2019; Freedman, 1999; LeCam, 1973; Panov and Spokoiny, 2015).”
Large ball probabilities, Gaussian comparison and anti-concentration (2017, methodology): “…written as Therefore, Again, the Pinsker inequality allows to upperbound the total variation distance between the Gaussian measures N (0 , Σ G ) and N ( a , Σ G 1 ) , however the answer is given via the Kullback-Leibler distance between these two measures: (1.7) see e.g. Panov and Spokoiny (2015).”