A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms

By James P. Hobert, Dobrin Marchev. 2008

92 citations recorded by the source corpus.

The data augmentation (DA) algorithm is a widely used Markov chain Monte Carlo (MCMC) algorithm that is based on a Markov transition density of the form p(x|x') = fy f X|Y (x|y) f Y|X (y|x')dy, where f X|Y and f Y|X are conditional densities. The PX-DA and marginal augmentation algorithms of Liu and Wu [J. Amer. Statist. Assoc. 94 (1999) 1264-1274] and Meng and van Dyk [Biometrika 86 (1999) 301-320] are alternatives to DA that often converge much faster and are only slightly more computationally demanding. The transition densities of these alternative algorithms can be written in the form p R (x|x') = f Y∫Y f X|Y (x|y')R(y,dy')f Y|X (y|x')dy, where R is a Markov transition function on Y. We prove that when R satisfies certain conditions, the MCMC algorithm driven by PR is at least as good as that driven by p in terms of performance in the central limit theorem and in the operator norm sense. These results are brought to bear on a theoretical comparison of the DA, PX-DA and marginal augmentation algorithms. Our focus is on situations where the group structure exploited by Liu and Wu is available. We show that the PX-DA algorithm based on Haar measure is at least as good as any PX-DA algorithm constructed using a proper prior on the group.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6487621819e836d86a544b91a192f7551026ae3e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

To Center or Not to Center: That Is Not the Question—An Ancillarity–Sufficiency Interweaving Strategy (ASIS) for Boosting MCMC Efficiency (2011, methodology): “The difference is that the kernel in the work of Hobert and Marchev (2008), using our notation, can be written as kHM(θ ′|θ) = ∫ ∫ p(θ ′|Y ′mis)p(Y ′mis|Ymis)p(Ymis|θ) dYmis dY ′mis, (2.18) where (θ ′, Y ′mis) and (θ, Ymis) have the same joint distribution; when Y ′mis = Ymis, (2.18) becomes the…”
Rao–Blackwellization for Bayesian Variable Selection and Model Averaging in Linear and Binary Regression: A Novel Data Augmentation Approach (2011, methodology): “Data augmentation with parameter expansion has been shown to improve convergence in many cases (Liu and Wu 1999; Meng and van Dyk 1999; Hobert and Marchev 2008). We experimented with several parameter expansion schemes based on scale changes for the latent data (not reported here), but did not see a”