Probabilistic Recurrent State-Space Models

By Andreas Doerr, Christian Daniel, Martin Schiegg, Duy Nguyen-Tuong, Stefan Schaal, Marc Toussaint, Sebastian Trimpe. 2018

149 citations recorded by the source corpus.

State-space models (SSMs) are a highly expressive model class for learning patterns in time series data and for system identification. Deterministic versions of SSMs (e.g. LSTMs) proved extremely successful in modeling complex time series data. Fully probabilistic SSMs, however, are often found hard to train, even for smaller problems. To overcome this limitation, we propose a novel model formulation and a scalable training algorithm based on doubly stochastic variational inference and Gaussian processes. In contrast to existing work, the proposed variational approximation allows one to fully capture the latent state temporal correlations. These correlations are the key to robust training. The effectiveness of the proposed PR-SSM is evaluated on a set of real-world benchmark datasets in comparison to state-of-the-art probabilistic model learning methods. Scalability and robustness are demonstrated on a high dimensional problem.

Subfields: Model Architectures, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Probabilistic & Bayesian Methods

Citations in the corpus

DeepMDP: Learning Continuous Latent Space Models for Representation Learning (2019, background): “For example, Chua et al. (2018); Doerr et al. (2018); Hafner et al. (2018); Buesing et al. (2018); Feinberg et al. (2018); Buckman et al. (2018) train models to predict a reward and a distribution over next states, minimizing the negative log-probability of the true next state.”
Learning Latent Dynamics for Planning from Pixels (2018, methodology): “Karl et al. (2016) address mode collapse to a single future by restricting the transition function, (Moerland et al., 2017) focus on multi-modal transitions, and Doerr et al. (2018) stabilize training of purely stochastic models.”
Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction (2019, background): “Non-Parametric State Space Models: Gaussian Process state space models (GP-SSMs) [34], [35] and variational approximations [36], provide an alternative non-parametric approach to forecasting non-linear state space models – mod-”
Dream to Control: Learning Behaviors by Latent Imagination (2019, background): “Learning effective latent dynamics models is becoming feasible through advances in deep learning and latent variable models (Krishnan et al., 2015; Karl et al., 2016; Doerr et al., 2018; Buesing et al., 2018).”
Mastering Atari with Discrete World Models (2020, methodology): “…et al., 2015; Karl et al., 2016; Chiappa et al., 2017; Babaeizadeh et al., 2017; Gemici et al., 2017; Denton and Fergus, 2018; Buesing et al., 2018; Doerr et al., 2018; Gregor and Besse, 2018), or combine the benefits of model-based and model-free approaches (Kalweit and Boedecker, 2017; Nagabandi…”
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies (2022, background): “(2), it is called a GP-NARX model (Doerr et al., 2018).”
A Survey on Bayesian Deep Learning (2016, methodology): “Control Embed to Control [125] LV ✓ Deep Variational Bayes Filters [57] LV ✓ Probabilistic Recurrent State-Space Models [19] LV ✓ Deep Planning Networks [34] LV ✓”
Interpretable Visual Reasoning via Probabilistic Formulation Under Natural Supervision (2020, background): “The ideas behind variational auto-encoders (VAEs) [33,47] have enabled complex latent dynamical systems like SVAE [29], non-linear SSMs [11,28,12,9] or parametrized deep Markov models [35,51].”
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model (2019, methodology): “…| x t +1 , z t , a t ) . in The log-likehood of the observations can then be bounded, similarly to the VAE bound Equation (5), as Prior work [16, 4, 8] has explored modeling such non-Markovian observation sequences, using methods such as recurrent neural networks with deterministic hidden state,…”