Variational Learning of Inducing Variables in Sparse Gaussian Processes

By Michalis K. Titsias. 2009

1,950 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “One interesting line of work, for example, would be to introduce a similar methodology by instead employing the sparse Gaussian process as the underlying probabilistic model (Snelson & Ghahramani, 2005; Titsias, 2009; Hensman et al., 2013).”
Zero-shot Knowledge Transfer via Adversarial Belief Matching (2019, methodology): “While early techniques used a subset of the training data as inducing points (Candela and Rasmussen, 2005), creating pseudo data using variational techniques was later shown more efficient (Titsias, 2009).”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, background): “In contrast, a variational approach has since been proposed to marginalize the pseudoinputs to maximize fidelity to the original exact GP [152] rather than the likelihood of the approximate GP.”
G-Net: A Deep Learning Approach to G-computation for Counterfactual Outcome Prediction Under Dynamic Treatment Regimes (2020, background): “Sparse GPs, which introduceM inducing points, have at leastO(M(2)N) [23] time complexity.”
Deep Gaussian Processes (2012, methodology): “Moreover, as in common sparse methods for Gaussian processes [Titsias, 2009], the complexity of each generative GP mapping is reduced from the typical O ( N 3 ) to O ( NM 2 ) .”
Variational Auto-Regressive Gaussian Processes for Continual Learning (2020, methodology): “By utilizing scalable sparse approximations for Gaussian processes (Titsias, 2009; Hensman et al., 2013; 2015) and advances in variational inference (Hoffman et al.”
Functional Regularisation for Continual Learning using Gaussian Processes (2019, methodology): “(3) The exact posterior distribution p(f1|u1,y1)p(u1|y1) in this augmented model is approximated by a distribution of the form (Titsias, 2009), q(f1,u1) = pθ(f1|u1)q(u1), where q(ui) is a variational distribution and pθ(f1|u1) is the GP prior conditional, pθ(f1|u1) = N (f1|KX1Z1K−1Z1 u1,KX1 −QX1).”
Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling (2018, methodology): “There are a wide variety of methods to approximate Gaussian processes using, for example, pseudo-observations (Snelson & Ghahramani, 2006) or variational inference (Titsias, 2009).”
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks (2017, methodology): “In this work to pick inducing points we use the variational method of Titsias [58].”
Stochastic Variational Deep Kernel Learning (2016, methodology): “In particular, a similar variational lower bound as in Eq (3) was proposed in [24, 10] for a sparse GP, which were extended to non-conjugate likelihoods, with the intractable integrals estimated using Gaussian quadrature as in the KLSP-GP [11] or univariate Gaussian samples as in the SAVI-GP [6].”
Gaussian Processes for Big Data (2013, methodology): “We start with a succinct rederivation of the variational approach to inducing variables of Titsias [2009].”
Scalable Variational Gaussian Process Classification (2014, background): “…the following inequality is used Substituting this bound on the conditional into the standard expression p ( y ) = (cid:82) p ( y | u ) p ( u )d u gives a tractable bound on the marginal likelihood for the Gaussian case [Titsias, 2009]: where σ 2 is the variance of the Gaussian likelihood term.”
Streaming Sparse Gaussian Process Approximations (2017, methodology): “Arguably the most important and influential approximations of this sort are pseudo-point approximation schemes that employ a set of M N pseudo-points to summarise the observational data thereby reducing computational costs to O(NM2) and O(M2) for training and prediction, respectively [2, 3].”
Meta Reinforcement Learning with Latent Variable Gaussian Processes (2018, methodology): “To address this we turn to the variational sparse GP approximation [30] and approximate the posterior GP with a variational distribution q ( f ( · )) that depends on a small set of M T inducing points .”
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, background): “Nonetheless, there is a broad literature on approximate Bayesian inference with GPs which can be utilized for efficient implementation [36, chapter 8]; [34, 38].”
GPflow: A Gaussian Process Library using TensorFlow (2016, methodology): “To this end we support ‘variationally sparse’ methods which ensure that the approximation is scalable and close (Opper and Archambeau, 2009), SGPR (Titsias, 2009), SVGP (Hensman et al., 2013, 2015b) and SGPMC (Hensman et al., 2015a).”
Few-shot learning of neural networks from scratch by pseudo example optimization (2018, background): “Titsias [33] suggests a variational approach which provides an objective function for optimizing inducing points.”
Continual Multi-task Gaussian Processes (2019, methodology): “(7), leading to where we extended the factorisation of Titsias (2009) to accept the entire function space f .”
Parametric Gaussian process regression for big data (2017, background): “However, as is truly pointed out in [16] even these reduced storage are prohibitive for “big data”.”