A Bernstein-Von Mises Theorem for discrete probability distributions

By Stéphane Boucheron, Elisabeth Gassiat. 2008

51 citations recorded by the source corpus.

We investigate the asymptotic normality of the posterior distri- bution in the discrete setting, when model dimension increases with sample size. We consider a probability mass function �0 on \{0} and a sequence of truncation levels (kn)n satisfying k 3 nninfikn �0(i). Let ˆ � denote the maximum likelihood estimate of (�0(i))ikn and letn(�0) denote the kn- dimensional vector which i-th coordinate is defined by p n(ˆ �n(i) �0(i)) for 1 � ikn. We check that under mild conditions on �0 and on the sequence of prior probabilities on the kn-dimensional simplices, after centering and rescaling, the variation distance between the posterior distribution recen- tered around ˆ �n and rescaled by p n and the kn-dimensional Gaussian dis- tribution N(�n(�0),I 1 (�0)) converges in probability to 0. This theorem can be used to prove the asymptotic normality of Bayesian estimators of Shannon and Renyi entropies. The proofs are based on concentration inequalities for centered and non- centered Chi-square (Pearson) statistics. The latter allow to establish pos- terior concentration rates with respect to Fisher distance rather than with respect to the Hellinger distance as it is commonplace in non-parametric Bayesian statistics.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems (2013, methodology): “available information in the data. Some recent results study the impact of a growing parameter dimension p n on the quality of Gaussian approximation of the posterior. We mention Ghosal (1999, 2000), Boucheron and Gassiat (2009), Johnstone (2010) for specic examples. See the discussion after Theorem”
Adaptive Bernstein–von Mises theorems in Gaussian white noise (2014, background): “For the case of finite-dimensional posteriors with increasing dimension, see Ghosal [19] and Bontemps [4] for the case of regression or Boucheron and Gassiat [5] for discrete probability distributions.”
The semiparametric Bernstein-von Mises theorem (2010, background): “Boucheron and Gassiat [4] consider the Bernstein–von Mises theorem for families of discrete distributions.”
Nonparametric Bernstein–von Mises theorems in Gaussian white noise (2012, background): “A number of BvM-type results have been obtained for the fixed finite-dimensional posterior with dimension increasing to infinity: Ghosal [16] and Bontemps [6] consider regression with a finite number of regressors, Ghosal [17] and Clarke and Ghosal [9] consider exponential families, and the case of ”
Bernstein - von Mises Theorem for growing parameter dimension (2013, background): “…(2000); Ghosal and van der Vaart (2007) for a general theory in the i.i.d. case; Ghosal (1999), Ghosal (2000) for high dimensional linear models; Boucheron and Gassiat (2009), Kim (2006) for non-Gaussian models; Shen (2002), Bickel and Kleijn (2012), and Castillo (2012) for the semiparametric…”
High dimensional Bernstein-von Mises: simple examples (2010, methodology): “Boucheron and Gassiat [2] develop a Bernstein-von Mises theorem for discrete probability distributions of growing dimension, and consider application to functionals such as Shannon and Renyi entropies.”
Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors (2010, methodology): “…particular, two situations have been studied: regression models in Ghosal (1999); exponential models in Ghosal (2000), Clarke and Ghosal (2010) and Boucheron and Gassiat (2009) (this last one deals with the discrete case, when the observations follow some unknown infinite multinomial…”
A Bernstein–von Mises theorem for smooth functionals in semiparametric models (2013, methodology): “In between semi- and nonparametric results, BvM for parameters with growing dimension have been obtained in, for example, Ghosal [21], Boucheron and Gassiat [7] and Bontemps [6].”
Critical dimension in the semiparametric Bernstein—von Mises theorem (2014, background): “In [5] the BvM result is proved in a specific class of i.”
BERNSTEIN-VON MISES THEOREM FOR LINEAR FUNCTIONALS OF THE DENSITY (2009, background): “In a discrete setting, [2] derive Bernstein-von Mises results, in particular satisfied by Dirichlet priors.”