SMEM Algorithm for Mixture Models

By Naonori Ueda, Ryohei Nakano, Zoubin Ghahramani, Geoffrey E. Hinton. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

How to Represent Part-Whole Hierarchies in a Neural Network (2021, methodology): “To make methods like EM work well when the number of clusters is unknown, it is helpful to introduce split and merge operations [Ueda et al., 2000] but these operations happen automatically during island formation.”
Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression (2007, methodology): “Extensive simulation and experimental results of the proposed algorithm on synthetic and real data are given in Section V.”
Online Model Selection Based on the Variational Bayes (2001, methodology): “Sequential model selection procedures (Ghahramani & Beal, 2000; Ueda, 1999) have also been proposed by combining the VB method and the split-and-merge algorithm ( Ueda, Nakano, Ghahramani, & Hinton, 1999 ).,As a mechanism for structural change, we adopt the split-and-merge method proposed by Ueda e”
Incremental learning of nonparametric Bayesian mixture models (2008, methodology): “Note that we can likely achieve greater speedup by initializing each learning round with the previous round’s model estimate and using a split-merge procedure [ 17 ], although we did not pursue this here.”
Non-linear Bayesian Image Modelling (2000, methodology): “However, in order to reduce the computational complexity of the inference problem we adopt an alternative approach based on model comparison using the numerically evaluated lower bound L(Q) which approximates the log model probability lnP (V ). Our optimization mechanism dynamically adapts the value”
Expectation-Maximization Gaussian-Mixture Approximate Message Passing (2012, methodology): “To select the worstfitting mixture component, one could use the approach of split-and-merge-EM [14], i.”
A Greedy EM Algorithm for Gaussian Mixture Learning (2002, methodology): “However, we decided to compare with the regular EM since the latter is still state of the art in mixture problems and also because several other algorithms report results compared to those of the regular EM [3, 17 ].”
Fast Approximate Energy Minimization with Label Costs (2010, background): “Another general heuristic is to fit a new model to the inliers of two existing models, and then add this new model to the candidate list; this ‘merge’ heuristic ( Ueda et al. 2000 ) gives energy (� � ) an opportunity to jump out of”
The Infinite Gaussian Mixture Model (1999, methodology): “EM [Ueda et al, 1998] and 4) it is much simpler to handle the infinite limit than to work with finite models with unknown sizes, as in [Richardson & Green, 1997] or traditional approaches based on extensive crossvalidation.”
Unsupervised segmentation of natural images via lossy data compression (2008, unknown): “complicated split-and-merge strategies [35].”
Feature Selection for Unsupervised Learning (2004, methodology): “Other interesting and current directions in feature selection involving feature transformations are mixtures of principal component analyzers (Kambhatla and Leen, 1997; Tipping and Bishop, 1999) and mixtures of factor analyzers (Ghahramani and Beal, 2000; Ghahramani and Hinton, 1996; Ueda et al., 19”
Incremental Online Learning in High Dimensions (2005, background): “Thus, adding a new data point in SVMR is computationally rather expensive, a property that is also shared by GPR. VBM suffers from similar problems due to the need for storing and reevaluating data when adding new mixture components (Ueda, Nakano, Ghahramani, & Hinton, 2000).”
Bayesian model search for mixture models based on optimizing variational bounds (2002, methodology): “To escape from such conﬁgurations, we employ the idea of the SMEM algorithm (Ueda et al., 2000) that we previously developed within the ML framework.”