Slice sampling covariance hyperparameters of latent Gaussian models

By Iain Murray, Ryan P. Adams. 2010

237 citations recorded by the source corpus.

The Gaussian process (GP) is a popular way to specify dependencies between random variables in a probabilistic model. In the Bayesian framework the covariance structure can be specified using unknown hyperparameters. Integrating over these hyperparameters considers different possible explanations for the data when making predictions. This integration is often performed using Markov chain Monte Carlo (MCMC) sampling. However, with non-Gaussian observations standard hyperparameter sampling approaches require careful tuning and may converge slowly. In this paper we present a slice sampling approach that requires little tuning while mixing well in both strong- and weak-data regimes.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Freeze-Thaw Bayesian Optimization (2014, methodology): “We follow the same procedure as in [4], using a Mat´ern kernel, an inferred constant prior mean, and integrating out the hyperparameters via slice sampling [5].”
Input Warping for Bayesian Optimization of Non-Stationary Functions (2014, methodology): “Following their treatment, we use the Mat´ern 5 / 2 kernel and we marginalize over kernel parameters θ using slice sampling [Murray and Adams, 2010].”
Entropy Search for Information-Efficient Global Optimization (2011, methodology): “re functions of kernel parameters, so the numerical values in this plot have relevance only relative to each other. Note the strong dierences in both mean and covariance functions of the posteriors. Murray and Adams (2010) for details. In the latter case, the belief p(f) over the function is a mixtu”
Gaussian Process Kernels for Pattern Discovery and Extrapolation (2013, methodology): “This marginal likelihood can be optimised to estimate hyperparameters such as (cid:96) , or used to integrate out the hyperparameters via Markov chain Monte Carlo (Murray & Adams, 2010).”
A Spectral Approach to Gradient Estimation for Implicit Distributions (2018, unknown): “As pointed out in previous works (Murray & Adams, 2010; Sejdinovic et al., 2014), sampling from the posterior of θ is challenging, e.g., Gibbs sampling often gets stuck due to p ( θ f , y ) is very sharp.”
Variational learning for latent Gaussian model of discrete data (2012, background): “, 2008], Gibbs sampling [Salakhutdinov and Mnih, 2008a], slice sampling [Murray and Adams, 2010], etc.”
Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters (2013, methodology): “SPEARMINT uses a Gaussian process (GP) to model pM(f | λ) and performs slice sampling over the GP’s hyperparameters [19].”
Bayesian Optimization with Unknown Constraints (2014, methodology): “When the posterior distribution cannot be computed in closed form due to a non-Gaussian likelihood, we use elliptical slice sampling (Murray et al., 2010) to sample g(x).”
Multi-Task Bayesian Optimization (2013, methodology): “As in [6] we use the Matérn 5/2 kernel and we marginalize over kernel parameters θ using slice sampling [9].”
Beyond Manual Tuning of Hyperparameters (2015, methodology): “SPEARMINT [61] uses a Gaussian process (GP) to model pMðf j kÞ and performs slice sampling over the GP’s hyperparameters [47].”
Bayesian Inference and Model Assessment for Spatial Point Patterns Using Posterior Predictive Samples (2017, background): “Murray et al. (2010) and Murray and Adams (2010) develop an elliptical slice sampling (ESS) algorithm for latent Gaussian fields and their hyperparameters.”
Pseudo-Marginal Bayesian Inference for Gaussian Processes (2013, methodology): “In [22], the SURR method achieved higher efﬁciency per covariance construction, likelihood evaluation and running time compared to the AA method.”