Online Sparse Multi-Output Gaussian Process Regression and Learning

By Le Yang, Ke Wang, Lyudmila S. Mihaylova. 2019

32 citations recorded by the source corpus.

This paper proposes an approach for online training of a sparse multi-output Gaussian process (GP) model using sequentially obtained data. The considered model combines linearly multiple latent sparse GPs to produce correlated output variables. Each latent GP has its own set of inducing points to achieve sparsity. We show that given the model hyperparameters, the posterior over the inducing points is Gaussian under Gaussian noise since they are linearly related to the model outputs. However, the inducing points from different latent GPs would become correlated, leading to a full covariance matrix cumbersome to handle. Variational inference is thus applied and an approximate regression technique is obtained, with which the posteriors over different inducing point sets can always factorize. As the model outputs are non-linearly dependent on the hyperparameters, a novel marginalized particle filer (MPF)-based algorithm is proposed for the online inference of the inducing point values and hyperparameters. The approximate regression technique is incorporated in the MPF and its distributed realization is presented. Algorithm validation using synthetic and real data is conducted, and promising results are obtained.

Subfields: Analytical Chemistry, Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

Yee Whye Teh

In Artificial Intelligence

Citations in the corpus

Continual Learning via Sequential Function-Space Variational Inference (2023, positive): “, 2017) and online sparse multi-output GP regression (Yang et al., 2019) built on the work of Csató and Opper (2002) and Csató (2002), and are effective approaches to continual learning for regression tasks.”
Continual Multi-task Gaussian Processes (2019, background): “The work by Yang et al. (2018) also derives tractable variational lower bounds based on the sparse approximation, but they do not handle non-Gaussian likelihoods and the learning method uses particle ﬁltering with a ﬁxed number of inducing points.”