Bayesian Modelling and Inference on Mixtures of Distributions

By Jean-Michel Marin, Kerrie L. Mengersen, Christian P. Robert. 2005

506 citations recorded by the source corpus.

Publisher Summary Mixture distributions comprise a finite or infinite number of components, possibly of different distributional types, that can describe different features of data. The Bayesian paradigm allows for probability statements to be made directly about the unknown parameters, prior or expert opinion to be included in the analysis, and hierarchical descriptions of both local-scale and global features of the model. This chapter aims to introduce the prior modeling, estimation, and evaluation of mixture distributions in a Bayesian paradigm. The chapter shows that mixture distributions provide a flexible, parametric framework for statistical modeling and analysis. Focus is on the methods rather than advanced examples, in the hope that an understanding of the practical aspects of such modeling can be carried into many disciplines. It also points out the fundamental difficulty in doing inference with such objects, along with a discussion about prior modeling, which is more restrictive than usual, and the constructions of estimators, which also is more involved than the standard posterior mean solution. Finally, this chapter gives some pointers to the related models and problems like mixtures of regressions and hidden Markov models as well as Dirichlet priors.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/13b6033bc1683592ce120b98612d21d593576d6c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparamatric estimation of the density (2010, background): “There is a vast literature on mixture models because of their rich structure which allows for different uses; for instance, they are well known to be adapted to the modelling of heterogeneous populations as is used, for example, in cluster analysis (for a good review on mixture models see [10] or [1”
A Survey of Statistical Network Models (2009, methodology): “This model combines the original latent space model [146] with a finite mixture of Gaussians approach to clustering [297; 205].”
Deviance information criteria for missing data models (2006, methodology): “(We refer the reader to Celeux et al. (2000) and Marin et al. (2005) for details of the now-standard implementation of an MCMC algorithm in mixture settings.)”
Harold Jeffreys’s Theory of Probability Revisited (2008, background): “In fact, mixture models do not allow for independent improper priors on their components (Marin, Mengersen and Robert, 2005).”
DENSITY ESTIMATION BY DUAL ASCENT OF THE LOG-LIKELIHOOD ∗ (2010, background): “In a parametric approach, one considers a family of probability densities depending on a set of parameters, and maximizes the likelihood of the observations in the allowed parameter range; Gaussian mixtures [3] and smoothing splines [9] are popular choices.”
Robust estimation in the normal mixture model based on robust clustering (2008, background): “On the other hand, the Markov chain Monte Carlo (MCMC) algorithms have given the possibility of treating the problem from a Bayesian point of view (an overview on mixtures in this Bayesian setting is Marin, Mengersen and Robert [15]).”
A Bayesian information criterion for singular models (2013, methodology): “We apply sBIC1 to a familiar example, namely the galaxies data set that has been discussed in detail in Aitkin (2001) and also in example 4 in Marin et al. (2005).”
Bimodal extension based on the skew‐normal distribution with application to pollen data (2011, background): “Motivation for considering such family of models is the lacking in the literature of parametric models that can more adequately replace mixtures of distributions which, as is quite well known, presents serious estimation problems either from classical or Bayesian approaches (McLachan and Peel, 2000;”
Testing hypotheses via a mixture estimation model (2014, background): “…demonstrated through the various examples in the paper, the setting is actually easier than with a standard mixture estimation problem (Diebolt and Robert, 1994; Marin et al., 2005) because of the existence of common parameters that allow for the original MCMC samplers to be turned into proposals.”
Skew mixture models for loss distributions: a Bayesian approach (2012, background): “…of the posterior distributions arising in this context, standard MCMC algorithms usually fail to explore all the support of the target posterior distribution even in the simple case of Gaussian component densities, see e.g. Robert and Casella (2004), Celeux et al. (2000) and Marin et al. (2005).”
Multivariate non-normally distributed random variables in climate research - introduction to the copula approach (2008, methodology): “multivariate Gaussian mixture models (Marin et al., 2005; McLachlan and Peel, 2000).”
Estimating the Marginal Likelihood Using the Arithmetic Mean Identity (2017, background): “There exist different Bayesian solutions for the problem (see, e.g. Celeux et al. (2000), Stephens (2000), Berkhof et al. (2003), Marin et al. (2005), Chung et al. (2004), Jasra et al. (2005), Frühwirth-Schnatter (2006)).”
LEMNA: Explaining Deep Learning based Security Applications (2018, methodology): “Given sufficient data samples, whether the classifier has a linear or non-linear decision boundary, the mixture regression model can nearly perfectly approximate the decision boundary (using a finite set of linear models) [35].”