Variational Bayesian Model Selection for Mixture Distributions

By Adrian Corduneanu, Charles M. Bishop. 2001

451 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Nonlinear Statistical Learning with Truncated Gaussian Graphical Models (2016, methodology): “Another approach is to use the mean-ﬁeld variational Bayesian (VB) method (Jordan et al., 1999; Corduneanu & Bishop, 2001), which approximates the true posterior p ( H | Y , X ) with a factorized distribution q ( H | (cid:101) Θ ) = =1 =1 q ( h i ( k ) | (cid:101) Θ ) , parameterized by (cid:101) Θ…”
A survey of uncertainty in deep neural networks (2021, background): “Notable examples include Bayesian model selection [109], [110], [111], [112], model compression [76], [113], [114], active learning [115], [23], [116], continual learning [117],”
The variational approximation for Bayesian inference (2008, methodology): “REMOVING THE PRIOR FROM THE MIXING WEIGHTS In [28], another example of a Bayesian GMM model has been proposed that does not assume a prior on the mixing weights {π j }, which are treated as parameters and not as random variables.”
Bounded Approximations for Marginal Likelihoods (2010, methodology): “L4: L4 is the estimate of lower bound of log marginal likelihood using the traditional variational method (Corduneanu and Bishop, 2001; Wang and Titterington, 2004).”
Bayesian reasoning and machine learning (2012, unknown): “See also [109, 67] for an approximate variational treatment focussed on Bayesian Gaussian mixture models.”
Regression Density Estimation With Variational Methods and Stochastic Approximation (2012, background): “Corduneanu and Bishop (2001) considered a variational lower bound on the log marginal likelihood with all parameters, except the mixing coefficients, integrated out, and estimated the mixing coefficients by maximizing the lower bound.”
Variational inference in nonconjugate models (2012, unknown): “As examples, it has been applied to Bayesian mixtures (Attias, 2000; Corduneanu and Bishop, 2001), factorial models (Ghahramani and Jordan, 1997), and probabilistic topic models (Blei et al., 2003).”
Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model (2006, background): “…Markov models (MacKay (1997)), graphical models (Attias (1999, 2000)), mixture models (Humphreys and Titterington (2000); Penny and Roberts (2000); Corduneanu and Bishop (2001); Ueda and Ghahramani (2003)), mixtures of factor analysers (Ghahramani and Beal (2000)) and state space models…”
Multivariate sharp quadratic bounds via -strong convexity and the Fenchel connection (2016, background): “of settings, including graphical models [13], mixture modelling [6, 19, 16, 18], and mixtures of experts [21].”
Variational approximations in Bayesian model selection for finite mixture distributions (2007, methodology): “As mentioned previously, Corduneanu and Bishop (2001) take an approach based on optimising the mixing co-efficients using type-2 maximum likelihood and marginalise out the model parameters using variational methods which leads to automatic recovery of the number of components.”
Pattern Recognition and Machine Learning (2006, unknown): “An alternative approach to determining a suitable value for K is to treat the mixing coefficients π as parameters and make point estimates of their values by maximizing the lower bound (Corduneanu and Bishop, 2001) with respect to π instead of maintaining a probability distribution over them as in t”
Recent Advances in Robot Learning from Demonstration (2020, methodology): “Regarding the use of Gaussian mixture models as policy functions, a more intuitive way to determine the optimal number of components is the use of Bayesian variational inference (209).”
Model-Based Clustering (2016, background): “Their fast and deterministic nature has made the variational Bayes approach increasingly popular over the past decade or two (e.g., Waterhouse, MacKay and Robinson 1996; Jordan et al. 1999; Corduneanu and Bishop 2001).”
The Controlled Thermodynamic Integral for Bayesian Model Comparison (2014, background): “, 2011) and variational approximations (Corduneanu and Bishop, 2001).”
A General Method for Robust Bayesian Modeling (2015, methodology): “…models, such as in Bayesian linear regression and logistic regression (Bishop, 2006), local latent variables, such as in Bayesian mixtures (Corduneanu and Bishop, 2001) and topic models (Blei et al., 2003; Blei, 2012), and non-exchangeable data, such as in a hidden Markov model (Rabiner,…”
A Generalized Methodology for Data Analysis (2018, background): “In reality, data distributions are usually multimodal [20]–[23], therefore the local description should be improved.”
Simultaneous feature selection and clustering using mixture models (2004, methodology): “Finally, principles other than MML, such as variational Bayes [12], can be adopted to perform model selection.”
Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations (2012, background): “Formerly, Corduneanu and Bishop (2001) veriﬁed through experiments and comparisons with cross-validation that the variational lower bound is a good score for model selection in Gaussian mixture models.”