Sparse Gaussian Processes using Pseudo-inputs

By Edward Snelson, Zoubin Ghahramani. 2005

2,039 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “One interesting line of work, for example, would be to introduce a similar methodology by instead employing the sparse Gaussian process as the underlying probabilistic model (Snelson & Ghahramani, 2005; Titsias, 2009; Hensman et al., 2013).”
Zero-shot Knowledge Transfer via Adversarial Belief Matching (2019, methodology): “Inducing point methods (Snelson and Ghahramani, 2005) were introduced to make Gaussian Processes (GP) more tractable.”
Zero-Shot Knowledge Distillation in Deep Networks (2019, methodology): “These pseudo examples are obtained using inducing point (Snelson & Ghahramani, 2006) method via iterative optimization technique in an adversarial manner which makes the training procedure complicated.”
Neural Architecture Search with Bayesian Optimisation and Optimal Transport (2018, methodology): “This expectation can be computed in closed form for GPs.”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “1) Sparse Pseudoinput Gaussian Processes (SPGPs): One early approach to modeling large n with GPs considered using m G n inducing pseudoinputs to reduce the rank of the covariance matrix to m, resulting in a significant reduction in computational cost [137], [140].”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, methodology): “We compute point estimates for the parameters using MAP estimation and the FITC sparse GP (Snelson and Ghahramani, 2006) implementation in PyMC3 (Salvatier et al.”
Set Transformer (2018, methodology): “…of letting trainable vectors I directly interact with data points is loosely based on the inducing point methods used in sparse Gaussian processes (Snelson & Ghahramani, 2005) and the Nystr¨om method for matrix decomposition (Fowlkes et al., 2004). m trainable inducing points can also be seen as…”
Conditional Neural Processes (2018, background): “Unfortunately, such Bayesian approaches quickly become computationally intractable as the dataset or dimensionality grows (Snelson & Ghahramani, 2006).”
Grammar Variational Autoencoder (2017, methodology): “We use these vectors and their corresponding property estimates to train a sparse Gaussian process (SGP) model with 500 inducing points (Snelson & Ghahramani, 2005), which is used to make predictions for the properties of new points in latent space.”
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules (2016, methodology): “We trained a sparse Gaussian process (GP) model [48] with 500 inducing points to predict the cost of each molecule from the molecule’s feature vector.”
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “A common approach to handle large datasets is to learn a small set of inducing points, which act as a proxy for the training data (Silverman, 1985; Quinonero-Candela & Rasmussen, 2005; Snelson & Ghahramani, 2006).”
Functional Regularisation for Continual Learning using Gaussian Processes (2019, methodology): “ a small set Z 1 = fz 1;jg M 1 j=1 of inducing inputs where each z 1;jlives in the same space as each training input x 1;j. The inducing set Z 1 can be a subset of X 1 or it can contain pseudo inputs [35], i.e. points lying between the training inputs. For simplicity next we consider Z 1 as pseudo p”
Approximate Inference Turns Deep Networks into Gaussian Processes (2019, methodology): “Figure 3: This figure shows a visualization of the predictive distributions on a modified version of the Snelson dataset [20].”
Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling (2018, methodology): “There are a wide variety of methods to approximate Gaussian processes using, for example, pseudo-observations (Snelson & Ghahramani, 2006) or variational inference (Titsias, 2009).”
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks (2017, background): “These can be fewer in number than the number of data points and optimised in position [58, 55, 4, 7] or be large in number but placed in a computationally e�cient structure [62, 60].”
Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks (2016, methodology): “For scalability, we used sparse GPs based on the FITC approximation [19], using a total of 150 inducing points.”
Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors (2016, methodology): “In order to fully utilize this property we adopt an idea from the GP literature: the concept of pseudo-data (Snelson & Ghahramani, 2005).”
Hidden physics models: Machine learning of nonlinear partial differential equations (2017, background): “This surprising performance is achieved at the cost of explicitly encoding the underlying physical laws expressed by the Navier-Stokes equations in the covariance functions of the hidden physics model (26).”
Scalable Variational Gaussian Process Classification (2014, methodology): “Here we focus on inducing point meth-ods [Snelson and Ghahramani, 2005], where the latent variables are augmented with additional input-output pairs Z , u , known as ‘inducing inputs’ and ‘inducing variables’.”