Elliptical slice sampling

By Iain Murray, Ryan P. Adams, David John Cameron MacKay. International Conference on Artificial Intelligence and Statistics, 2009

505 citations recorded by the source corpus.

Many probabilistic models introduce strong dependencies between variables using a latent multivariate Gaussian distribution or a Gaussian process. We present a new Markov chain Monte Carlo algorithm for performing inference in models with multivariate Gaussian priors. Its key properties are: 1) it has simple, generic code applicable to many models, 2) it has no free parameters, 3) it works well for a variety of Gaussian process based models. These properties make our method ideal for use while model building, removing the need to spend time deriving and tuning updates for more complex algorithms.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/4c3d7bb6bcaa1e49dab561f1a74f236a7d100d68

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Zoubin Ghahramani
3. Andrew Gordon Wilson
4. Ruslan Salakhutdinov
5. Yoshua Bengio
6. Richard E. Turner
7. Quoc V. Le
8. Eric P. Xing

In Statistics and Probability

1. Zoubin Ghahramani
2. Jasper Snoek
3. Andrew Gordon Wilson
4. Ryan P. Adams
5. Mark A. Girolami
6. Michalis K. Titsias
7. Richard E. Turner
8. Maurizio Filippone

Citations in the corpus

• Scalable Inference for Gaussian Process Models with Black-Box Likelihoods (2015, background): “We note that [6] reported that automated variational inference (the focus of this paper) produced practically indistinguishable distributions (but run order of magnitude faster) when compared to sampling methods such as Elliptical Slice Sampling [5].”
• Discovering Latent Network Structure in Point Process Data (2014, methodology): “We use Elliptical Slice Sampling (Murray et al., 2010) to sample the conditional distribution of the vector y .”
• Unbiased Markov chain Monte Carlo methods with couplings (2020, methodology): “Gibbs sampling is another popular class of MCMC algorithms, in which components of a Markov chain are updated alternately by sampling from the target conditional distributions (chapter 10 of Robert and Casella (2004)), implemented for example in the software packages JAGS (Plummer et al.”
• Subspace Inference for Bayesian Deep Learning (2019, methodology): “, 2011) or elliptical slice sampling (ESS) (Murray et al., 2010).”
• Deep Convolutional Inverse Graphics Network (2015, background): “Neal [4] and Murray et al. [21] studied generative models with Gaussian priors having zero mean and arbitrary covariance structure.”
• Gaussian Process Regression Networks (2011, methodology): “For MCMC it is straightforward to do this by exponentiating the weights, as in Adams and Stegle (2008) and Adams et al. (2010).”
• Variational Heteroscedastic Gaussian Process Regression (2011, methodology): “The latter is validated experimentally through a comparison with the elliptical slice sampling MCMC method (Murray et al., 2010).”
• Bayesian Optimization and Semiparametric Models with Applications to Assistive Technology (2014, methodology): “GP hyperparameters for the classifier are sampled according to the elliptical slice sampling strategy of Murray et al. [2010]. An outline of the algorithm used to select the next point at which to evaluate the function being optimized is provided in Algorithm 3.”
• Deep Generative Vision as Approximate Bayesian Computation (2014, background): “[16] studied generative models with Gaussian priors having zero mean and arbitrary covariance structure.”
• Edinburgh Research Explorer Continuous Relaxations for Discrete Hamiltonian Monte Carlo (unknown): “Continuous problems are appealing because the gradient is on your side: Unlike discrete probability mass functions, in the continuous setting, densities have derivatives, contours, and curvature that can be used to inform sampling algorithms [6, 16, 18, 20, 27].”
• Picture: A probabilistic programming language for scene perception (2015, background): “To adaptively propose to a large set of latent variables at once, our inference library supports elliptical slice moves with or without adaptive step sizes (see Figure 2 for an example)[4, 33].”
• MCMC for Variationally Sparse Gaussian Processes (2015, methodology): “For a fixed θ, sampling v is relatively straightforward, and this can be done efficiently using HMC [12, 26, 27] or Elliptical Slice Sampling [28].”
• Copulas in Machine Learning (2013, methodology): “• A Markov Chain Monte Carlo technique to sample from this posterior, specifically the elliptical slice sampling method [29].”
• Approximate Gaussian Integration using Expectation Propagation (2011, methodology): “We note that Monte Carlo methods are not particularly well suited to this problem: rejection samplers from p0(x), importance samplers over the polyhedron A (“pinball” or “hit-and-run” MCMC algorithms (Herbrich, 2002)), and elliptical slice sampling (Murray et al., 2010) do not perform favorably (the”
• Pseudo-Marginal Bayesian Inference for Gaussian Processes (2013, methodology): “ELL-SS is based on an adaptation of the Slice Sampling algo-rithm [32] to propose new values of the latent variables.”
• Bayesian Inference and Model Assessment for Spatial Point Patterns Using Posterior Predictive Samples (2017, background): “Work by Sherman and Carlstein (1994), Lahiri (1999), and Lahiri (2003) suggests letting the shape of Bk mimic the shape of D.”