Bayes and empirical Bayes : Do they merge?

By Sonia Petrone, Judith Rousseau, Catia Scricciolo. 2012

82 citations recorded by the source corpus.

Bayesian inference is attractive for its coherence and good frequentist properties. However, eliciting a honest prior may be difficult and a common practice is to take an empirical Bayes approach, using some empirical estimate of the prior hyperparameters. Despite not rigorous, the underlying idea is that, for sufficiently large sample size, empirical Bayes leads to similar inferential answers as a proper Bayesian inference. However, precise mathematical results seem missing. In this work, we give more rigorous results in terms of merging of Bayesian and empirical Bayesian posterior distributions. We study two notions of merging: Bayesian weak merging and frequentist merging in total variation. We also show that, under regularity conditions, empirical Bayes asymptotically gives an oracle selection of the prior hyperparameters. Examples include empirical Bayes density estimation with Dirichlet process mixtures.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/fc708347109c16c96bbc949ffa09092762b0360a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kolyan Ray
2. Botond Szab'o
3. Judith Rousseau
4. Richard Nickl
5. I. Castillo
6. A. van der Vaart
7. Subhashis Ghosal
8. Christian P. Robert

In Management Science and Operations Research

1. Botond Szab'o
2. Judith Rousseau

In Statistics and Probability

1. Botond Szab'o
2. Kolyan Ray
3. Judith Rousseau
4. Richard Nickl
5. A. van der Vaart
6. I. Castillo
7. Subhashis Ghosal
8. Christian P. Robert

Citations in the corpus

• Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator (2015, methodology): “[22] have studied the asymptotic behaviour of the posterior distribution for general empirical Bayes approaches; they provide conditions to obtain consistency of the empirical Bayes posterior and in the case of parametric models characterized the behaviour of the maximum marginal likelihood estimato”
• On the Frequentist Properties of Bayesian Nonparametric Methods (2016, methodology): “A similar result was obtained in the parametric framework by (Petrone et al., 2014). www.annualreviews.org • 17 SUMMARY POINTS 1.”
• Adaptive Bernstein–von Mises theorems in Gaussian white noise (2014, background): “imple corollary to Theorems3.1and3.2, we have that the rescaled posteriors merge weakly 10 (with respect to weak convergence on H()) in the sense of Diaconis and Freedman [19]. By Proposition 2.1 of [41], we immediately have that the unscaled posteriors merge weakly with respect to the ‘ 2-topology ”