A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model

By Sonia Jain, Radford M. Neal. 2004

549 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6305dcc03c8378e371e73b0a68ff29f1167a65f0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, methodology): “The conventional inference methods for mixture models mainly consist of MCMC sampling [29, 30] and variational inference [31].”
The Indian Buffet Process: An Introduction and Review (2011, methodology): “The Chinese restaurant process (CRP) was named by Jim Pitman and Lester Dubins, based upon a metaphor in which the objects are customers in a restaurant, and the classes are the tables at which they sit (the process first appears in Aldous 1985, where it is attributed to Pitman, although .”
Modelling Relational Data using Bayesian Clustered Tensor Factorization (2009, methodology): “First, given the parameters θ, the chain updates c = (crel, cobj) using a collapsed Gibbs sampling sweep and a step of the split-and-merge algorithm (where the launch state was obtained with two sweeps of Gibbs sampling starting from a uniformly random cluster assignment) [5].”
Reconciling meta-learning and continual learning with online mixtures of tasks (2018, methodology): “In the stochastic setting, access to past data is unavailable, and so none of the standard algorithms and heuristics for inference in non-parametric models are applicable [ e.g., 26, 25].”
Infinite latent feature models and the Indian buffet process (2005, methodology): “Algorithms that go beyond the local changes in class assignments allowed by a Gibbs sampler are given by Jain and Neal (2004) and Dahl (2003).”
Fast Bayesian Inference in Dirichlet Process Mixture Models (2011, background): “…literature on Markov chain Monte Carlo (MCMC) algorithms for posterior computation, proposing marginal Gibbs sampling (MacEachern 1994; West, Müller, and Esco-bar 1994; Bush and MacEachern 1996), conditional Gibbs sampling (Ishwaran and James 2001), and split-merge (Jain and Neal 2004) algorithms.”
Parallel inference of dirichlet process Gaussian mixture models for unsupervised acoustic modeling: a feasibility study (2015, methodology): “Some of them [30, 31] are based on sampling using a Markov chain Monte Carlo (MCMC) scheme while others are based on variational inference [32–34].”
Multiple Hypothesis Testing by Clustering Treatment Effects (2007, methodology): “with a merge-split of Jain and Neal (2004) or Dahl (2003).”
A Tutorial on Bayesian Nonparametric Models (2011, unknown): “Neal (2000) provides an excellent survey of Gibbs sampling and other MCMC algorithms for inference in CRP mixture models (see also Escobar and West, 1995; Rasmussen, 2000; Ishwaran and James, 2001; Jain and Neal, 2004; Fearnhead, 2004; Wood and Griffiths, 2007).”
Multiscale Community Blockmodel for Network Exploration (2011, methodology): “One solution to this problem is a split-merge strategy, similar to that used by Jain and Neal (2004) for the vanilla Chinese Restaurant Process.”
The Nested Dirichlet Process (2008, methodology): “…in Bayesian statistics, due mainly to the availability of efficient computational techniques (Escobar and West 1995; Lo, Brunner, and Chan 1996; MacEachern and Müller 1998; Neal 2000; Jain and Neal 2000; Ishwaran and James 2001, 2003; Roberts and Papaspiliopoulos 2008; Blei and Jordan 2006).”
Interactive POMDPs with finite-state models of other agents (2017, background): “This problem is not new in the context of Gibbs sampling for mixture models [7,30].”
Model-Based Clustering (2016, methodology): “Although beyond the scope of this review, there has been much work on Dirichlet process mixtures (e.g., Jain and Neal 2004; Bdiri, Bougouli, and Ziou 2016) and this is sure to continue.”
Learning Systems of Concepts with an Infinite Relational Model (2006, methodology): “Since we integrate out η, inference can be carried out using Markov chain Monte Carlo methods to sample from the posterior on cluster assignments P (z|R) ∝ P (R|z)P (z) (Jain & Neal 2004), or by searching for the mode of this distribution.”
Bayesian infinite mixture model based clustering of gene expression profiles (2002, background): “Potential bene fits of running several shorter runs instead of a single long Gibbs sampler, as well as advantages of using potentially more efficient Gibbs sampling scheme (e.g. Jain and Neal, 2000; MacEachern, 1994 ) need to be further investigated.”
Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models (2007, methodology): “…Monte Carlo schemes do exist and include the sequential samplers of Liu (1996) and Ishwaran & James (2003), and referred to as the blocked Gibbs sampler in those papers, the particle ﬁlter of Fearnhead (2004) and the reversible jump method of Green & Richardson (2001) and Jain & Neal (2004).”
Revisiting k-means: New Algorithms via Bayesian Nonparametrics (2011, methodology): “ﬁnite mixture models, but several effective posterior inference techniques have been developed, including Gibbs sampling algorithms [16, 22], MetropolisHastings algorithms [16], split-merge proposals [11], variational inference [3], and permutation-augmented samplers [13]. 1 arXiv:1111.0352v1 [cs.LG”