Nonparametric Bayesian model selection and averaging

By Subhashis Ghosal, Jüri Lember, A. van der Vaart. 2008

101 citations recorded by the source corpus.

We consider nonparametric Bayesian estimation of a probability density p based on a random sample of size n from this density using a hierarchical prior. The prior consists, for instance, of prior weights on the regularity of the unknown density combined with priors that are appropriate given that the density has this regularity. More generally, the hierarchy consists of prior weights on an abstract model index and a prior on a density model for each model index. We present a general theorem on the rate of contraction of the resulting posterior distribution as n?8, which gives conditions under which the rate of contraction is the one attached to the model that best approximates the true density of the obser- vations. This shows that, for instance, the posterior distribution can adapt to the smoothness of the underlying density. We also study the posterior distribution of the model index, and find that under the same conditions the posterior distribution gives negligible weight to models that are bigger than the optimal one, and thus selects the optimal model or smaller models that also approximate the true density well. We apply these result to log spline density models, where we show that the prior weights on the regularity index interact with the priors on the models, making the exact rates depend in a complicated way on the priors, but also that the rate is fairly robust to specification of the prior weights.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/33b4ede953bfaafd7afe28affa2d720f5972032f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Bayesian Goodness of Fit Testing with Mixtures of Triangular Distributions (2009, methodology): “Very recently Ghosal et al. (2008) have provided sufficient conditions for consistency of the Bayes factor when F0 is parametric.”
BERNSTEIN-VON MISES THEOREM FOR LINEAR FUNCTIONALS OF THE DENSITY (2009, result): “Also related to this problem are the works of [13] and [10] who derive a general framework to obtain adaptive posterior concentration rates, the former applies her results to the Haar basis case.”
Posterior concentration rates for infinite dimensional exponential families (2012, result): “[9] who derive a general framework to obtain adaptive posterior concentration rates, the former applies her results to the Haar basis case.”
Catching up faster by switching sooner: a predictive approach to adaptive estimation with an application to the AIC–BIC dilemma (2012, background): “Ghosal et al. [2008] analyse exponential families defined on X = [0, 1].”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “Besides practical benefit, this correspondence provides a theoretical tool to analyze the hierarchical Bayes method, which need not rely on testing arguments (as in [12, 13, 34]).”
Two-sample Bayesian Nonparametric Hypothesis Testing (2009, background): “Consistency issues are discussed by Dass and Lee (2004), Rousseau (2007), Ghosal et al. (2008) and McVinish et al. (2009).”