Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models

By Omiros Papaspiliopoulos, Gareth O. Roberts. 2007

405 citations recorded by the source corpus.

Inference for Dirichlet process hierarchical models is typically performed using Markov chain Monte Carlo methods, which can be roughly categorized into marginal and conditional methods. The former integrate out analytically the infinite-dimensional component of the hierarchical model and sample from the marginal distribution of the remaining variables using the Gibbs sampler. Conditional methods impute the Dirichlet process and update it as a component of the Gibbs sampler. Since this requires imputation of an infinite-dimensional process, implementation of the conditional method has relied on finite approximations. In this paper, we show how to avoid such approximations by designing two novel Markov chain Monte Carlo algorithms which sample from the exact posterior distribution of quantities of interest. The approximations are avoided by the new technique of retrospective sampling. We also show how the algorithms can obtain samples from functionals of the Dirichlet process. The marginal and the conditional methods are compared and a careful simulation study is included, which involves a non-conjugate model, different datasets and prior specifications. Copyright 2008, Oxford University Press.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/1e40d280daae88a5f9063b5ebd80156b7b55c61e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Slice sampling mixture models (2011, methodology): “Later, for the illustrations and comparison with the other ‘conditional’ algorithms, namely the blocked Gibbs sampler (Ishwaran and James 2001) and the retrospective sampler (Papaspiliopoulos and Roberts 2008), we will consider three types of slice sampler.”
A Bayesian Semiparametric Approach to Intermediate Variables in Causal Inference (2011, methodology): “Using these concepts, Robins and Greenland (1992) and Pearl (2001) gave definitions for controlled direct effects and natural direct and indirect effects based on hypothetical interventions on the intermediate variable.”
Bayesian Inference Methods for Univariate and Multivariate GARCH Models: A Survey (2014, positive): “In order to perform Bayesian inference on the new model, the authors employ a stick-breaking sampling scheme and make use of the ideas proposed in Walker (2007), Papaspiliopoulos & Roberts (2008) and Pa-paspiliopoulos (2008).”
Compound random measures and their use in Bayesian non‐parametrics (2014, methodology): “This usually leads to computationally tractable methods which either extend random truncation methods such as retrospective sampling [38] or slice sampling [24], or develop truncation ideas for Dirichlet process mixtures [22].”
Tree-Structured Stick Breaking for Hierarchical Data (2010, methodology): “To avoid this difficulty, we use a slice sampling approach that can be viewed as a combination of the Dirichlet slice sampler of Walker [20] and the retrospective sampler of Papaspiliopolous and Roberts [21].”
Exact Bayesian inference in spatiotemporal Cox processes driven by multivariate Gaussian processes (2015, background): “ritten w.r.t. dominating measures: LK, L and Ldθ, respectively, in accordance with the dominating measure used in (9). Block β−K is inﬁnite-dimensional and, therefore, is sampled retrospectively (see Papaspiliopoulos and Roberts, 2008). This means that it is sampled at a ﬁnite number of locations as”
Bayesian Nonparametric Calibration and Combination of Predictive Distributions (2015, methodology): “…used in many fields due to the availability of efficient algorithms for posterior computations (Escobar and West 1995; MacEachern and Müller 1998; Papaspiliopoulos and Roberts 2008; © American Statistical Association Taddy 2010), including but not limited to applications in time series analysis…”
Bayesian inference with dependent normalized completely random measures (2014, background): “, the slice sampler [38] and the retrospective sampler [31].”
PhyloBayes MPI: phylogenetic reconstruction with infinite mixtures of profiles in a parallel environment. (2013, methodology): “In addition, we developed a hybrid between Gibbs-sampling and Metropolis–Hastings, inspired by (Papaspiliopoulos and Roberts 2008), for resampling the allocations of the sites to the components of the mixture.”
Joint Species Distribution Modeling: Dimension Reduction Using Dirichlet Processes (2017, methodology): “It is widely used in the Bayesian nonparametric literature as a prior for distributions rather than using customary parametric distributions adopting priors for the parameters (Escobar and West, 1995; Ishwaran and James, 2001; Papaspiliopoulos and Roberts, 2008).”
POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES (2011, methodology): “To estimate the model, we develop an MCMC algorithm based on slice sampling (Neal, 2003; Walker, 2007) and retrospective sampling (Papaspiliopoulos and Roberts, 2008; Papaspiliopoulos, 2008).”
Stick-Breaking Policy Learning in Dec-POMDPs (2015, methodology): “To accommodate an unbounded number of nodes, we apply the retrospective representation of SB priors [Papaspiliopoulos and Roberts, 2008] to the Dec-SBPR.”
A Sticky HDP-HMM With Application to Speaker Diarization (2009, methodology): “The beam sampler was inspired by a related approach for DP mixture models (Walker, 2007), which is conceptually similar to retrospective sampling methods (Papaspiliopoulos and Roberts, 2008).”
Stick-breaking autoregressive processes (2011, methodology): “We update si using a retrospective sampler (see Papaspiliopoulos and Roberts, 2008).”
Efficient MCMC for Climate Model Parameter Estimation: Parallel Adaptive Chains and Early Rejection (2012, methodology): “It was employed in (Beskos et al. 2006a; Papaspiliopoulos and Roberts 2008) under the title Retrospective Simulation for certain diﬀusion processes, in order to implement in ﬁnite time a simulation for inﬁnite dimensional random variables.”