Pseudo-Marginal Bayesian Inference for Gaussian Processes
By Maurizio Filippone, Mark A. Girolami. 2013
66 citations recorded by the source corpus.
The main challenges that arise when adopting Gaussian process priors in probabilistic modeling are how to carry out exact Bayesian inference and how to account for uncertainty on model parameters when making model-based predictions on out-of-sample data. Using probit regression as an illustrative working example, this paper presents a general and effective methodology based on the pseudo-marginal approach to Markov chain Monte Carlo that efficiently addresses both of these issues. The results presented in this paper show improvements over existing sampling methods to simulate from the posterior distribution over the parameters defining the covariance function of the Gaussian Process prior. This is particularly important as it offers a powerful tool to carry out full Bayesian inference of Gaussian Process based hierarchic statistical models in general. The results also demonstrate that Monte Carlo based integration of all model parameters is actually feasible in this class of models providing a superior quantification of uncertainty in predictions. Extensive comparisons with respect to state-of-the-art probabilistic classifiers confirm this assertion.
Subfields: Artificial Intelligence, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dino Sejdinovic
- Mark A. Girolami
- Chris J. Oates
- Arthur Gretton
- Heiko Strathmann
- Nicolas Chopin
- Omiros Papaspiliopoulos
- Andrew M. Stuart
In Statistics and Probability
- Mark A. Girolami
- Chris J. Oates
- Dino Sejdinovic
- Nicolas Chopin
- Heiko Strathmann
- Arthur Gretton
- Qiang Liu
- Andrew M. Stuart
Citations in the corpus
- Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families (2015, neutral): “in Bayesian Gaussian Process classification [4].”
- Control functionals for Monte Carlo integration (2014, background): “The marginalization of hyperparameters is a common problem in spatial statistics and Bayesian statistics in general (Besag and Green, 1993; Agapiou et al., 2014; Filippone and Girolami, 2014).”
- Kernel Adaptive Metropolis-Hastings (2013, background): “We consider the following nonlinear targets: (1) the posterior distribution of Gaussian Process (GP) classification hyperparameters (Filippone & Girolami, 2014) on the UCI glass dataset, and (2) the synthetic banana-shaped distribution ofHaario et al. (1999) and a flower-shaped disribution…”
- Analysis of the Gibbs Sampler for Hierarchical Inverse Problems (2013, methodology): “ an option. The so-called pseudo-marginal algorithm [5], might be an alternative for non-linear problems, and has been recently employed to perform Bayesian inference using Gaussian process priors in [18]. An interesting research direction is the comparison of the performance of the two MwG algorith”
- Hyperpriors for Matérn fields with applications in Bayesian inversion (2016, methodology): “Alternatives for Metropolis-within-Gibbs include e.g. pseudomarginal approach to MCMC studied by Filippone and Girolami 2014 [15].”