Variational Gaussian Process Dynamical Systems

By Andreas C. Damianou, Michalis K. Titsias, Neil D. Lawrence. 2011

115 citations recorded by the source corpus.

High dimensional time series are endemic in applications of machine learning such as robotics (sensor data), computational biology (gene expression data), vision (video sequences) and graphics (motion capture data). Practical nonlinear probabilistic approaches to this data are required. In this paper we introduce the variational Gaussian process dynamical system. Our work builds on recent variational approximations for Gaussian process latent variable models to allow for nonlinear dimensionality reduction simultaneously with learning a dynamical prior in the latent space. The approach also allows for the appropriate dimensionality of the latent space to be automatically determined. We demonstrate the model on a human motion capture data set and a series of high resolution video sequences.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Control and Systems Engineering

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “…2004; Barber and Chiappa 2006; Archambeau et al. 2007a, 2007b; Johnson and Willsky 2014; Foti et al. 2014), robust models (Tipping and Lawrence 2005; Wang and Blei 2015), and Gaussian process models (Titsias and Lawrence 2010; Damianou, Titsias, and Lawrence 2011; Hensman, Fusi, and Lawrence 2013).”
Deep Gaussian Processes (2012, methodology): “Because this type of deep GP only contains one hidden layer, it is identical to the model developed by [Damianou et al., 2011] (where the input given at the top layer of their model was a time vector, but their code is trivially generalized).”
Gaussian Processes for Big Data (2013, methodology): “Several GP based models involve inference of X , such as the GP latent variable model [Lawrence, 2005, Titsias and Lawrence, 2010] and its extensions [e.g. Damianou et al., 2011, 2012].”
Manifold Relevance Determination (2012, methodology): “We achieve this by building on recent variational approximations for standard GP-LVMs (Titsias & Lawrence, 2010; Damianou et al., 2011).”
Doubly Stochastic Variational Inference for Deep Gaussian Processes (2017, methodology): “This approach was extended in Damianou et al. (2011) to perform approximate inference in the model of Lawrence and Moore (2007), and shortly afterwards in a similar model Lázaro-Gredilla (2012), which also included a linear mean function.”
Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations (2013, background): “The extensive work on scaling inference in these Gaussian process models [23, 26, 27] provides several avenues for relaxing some of the approximations that we made in this work (while adding new ones).”
Variational Auto-encoded Deep Gaussian Processes (2015, methodology): “Inspired by deep neural networks, a deep Gaussian process instead employs process composition (Lawrence & Moore, 2007; Damianou et al., 2011; L ´ azaro-Gredilla, 2012; Damianou & Lawrence, 2013; Hensman & Lawrence, 2014).”
Expectation Propagation in Gaussian Process Dynamical Systems (2012, methodology): “More recently, variational [2] and EM-based [19] approaches for learning GPDS were proposed.”
Variational inference of latent state sequences using Recurrent Networks (2014, background): “Arguably the use of distributions for x t which have many different maxima and allow higher-order correlations in the individual components is promising.”