On posterior contraction of parameters and interpretability in Bayesian mixture modeling

By Aritra Guha, Nhat Ho, XuanLong Nguyen. Bernoulli, 2019

67 citations recorded by the source corpus.

We study posterior contraction behaviors for parameters of interest in the context of Bayesian mixture modeling, where the number of mixing components is unknown while the model itself may or may not be correctly specified. Two representative types of prior specification will be considered: one requires explicitly a prior distribution on the number of mixture components, while the other places a nonparametric prior on the space of mixing distributions. The former is shown to yield an optimal rate of posterior contraction on the model parameters under minimal conditions, while the latter can be utilized to consistently recover the unknown number of mixture components, with the help of a fast probabilistic post-processing procedure. We then turn the study of these Bayesian procedures to the realistic settings of model misspecification. It will be shown that the modeling choice of kernel density functions plays perhaps the most impactful roles in determining the posterior contraction rates in the misspecified situations. Drawing on concrete posterior contraction rates established in this paper we wish to highlight some aspects about the interesting tradeoffs between model expressiveness and interpretability that a statistical modeler must negotiate in the rich world of mixture modeling.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b0891118c10b8f10e6c57b9b5148247783e13535

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wenhao Ding
2. Mengdi Xu
3. Jiacheng Zhu
4. Baiming Chen
5. Ding Zhao
6. Zuxin Liu

In Statistics and Probability

1. Wenhao Ding
2. Mengdi Xu
3. Jiacheng Zhu
4. Baiming Chen
5. Ding Zhao
6. Zuxin Liu

Citations in the corpus

• Consistency of mixture models with a prior on the number of components (2022, unknown): “Elegant results have also been provided showing posterior consistency and contraction rates for estimation of the discrete mixing distribution (Nguyen, 2013) when using either class of models, as well as consistency for the number of components (Guha et al., 2021).”
• Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, methodology): “Other methods for merging experts can be found in [41].”