BLACK BOX VARIATIONAL INFERENCE FOR STATE SPACE MODELS

By Evan Archer, Il Memming Park, Lars Buesing, John P. Cunningham, Liam Paninski. 2015

175 citations recorded by the source corpus.

Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists across time. A few highly-structured models, such as the linear dynamical system with linear-Gaussian observations, have closed-form inference procedures (e.g. the Kalman Filter), but this case is an exception to the general rule that exact posterior inference in more complex generative models is intractable. Consequently, much work in time-series modeling focuses on approximate inference procedures for one particular class of models. Here, we extend recent developments in stochastic variational inference to develop a `black-box' approximate inference technique for latent variable models with latent dynamical structure. We propose a structured Gaussian variational approximate posterior that carries the same intuition as the standard Kalman filter-smoother but, importantly, permits us to use the same inference approach to approximate the posterior of much more general, nonlinear latent variable generative models. We show that our approach recovers accurate estimates in the case of basic models with closed-form posteriors, and more interestingly performs well in comparison to variational approaches that were designed in a bespoke fashion for specific non-conjugate models.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

Learning and Querying Fast Generative Models for Reinforcement Learning (2018, background): “…models We build directly on a plethora of recent work exploring the continuum of models ranging from standard recurrent neural networks (RNNs) to fully stochastic models with uncertainty (Chung et al., 2015; Archer et al., 2015; Fraccaro et al., 2016; Krishnan et al., 2015; Gu et al., 2015).”
Composing graphical models with neural networks for structured representations and fast inference (2016, background): “[20] the authors also consider the problem of variational inference in general continuous state space models but focus on using a structured Gaussian variational family without considering parameter learning.”
A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning (2017, background): “The VAE framework is a common workhorse in the stable of probabilistic inference methods, and it is extended to the temporal setting by [2, 6, 8, 16, 19].”
Sequential Neural Models with Stochastic Layers (2016, methodology): “The works by [1, 20] (Figure 4c) show that it is possible to improve inference in SSMs by using ideas from VAEs, similar to what is done in the stochastic part (the top layer) of SRNN.”
Structured Inference Networks for Nonlinear State Space Models (2016, background): “Bayer and Osendorfer and Fabius and van Amersfoort create a stochastic variant of RNNs by making the hidden state of the RNN at every time step be a function of independently sampled latent variables.”
Learning Awareness Models (2018, background): “…sequence-to-sequence modelling (Downey et al., 2017; Venkatraman et al., 2017; Chung et al., 2015; Fraccaro et al., 2016; Bayer & Osendorfer, 2014; Archer et al., 2015; Krishnan et al., 2015), particularly in the context of predicting distributions and in dynamics modelling in reinforcement…”
State Space LSTM Models with Particle MCMC Inference (2017, background): “The idea of parameterizing them with neural networks can be found in Ghahramani & Roweis (1999), as well as many recent works (Krishnan et al., 2015; Archer et al., 2015; Johnson et al., 2016; Krishnan et al., 2017; Karl et al., 2017) thanks to the development of recognition networks (Kingma &…”
Black-Box Variational Inference for Stochastic Differential Equations (2018, background): “Variational inference Several authors have looked at variational inference for SDEs (Archambeau et al., 2008) or re-lated problems such as Markov jump processes (Ruttor et al., 2010) and state space models (Archer et al., 2016; Quiroz et al., 2018).”
Variational Sequential Monte Carlo (2017, background): “Archer et al. [2015], Johnson et al. [2016] develop variational inference for state space models with conjugate dynamics, while Krishnan et al.”
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model (2019, background): “Several previous works have explored various modeling choices to learn stochastic sequential models [40, 4, 34, 16, 17, 12, 20].”
Creativity: Generating Diverse Questions Using Variational Autoencoders (2017, background): “[3] how to combine VAEs with continuous state-space models.”
Temporal Difference Variational Auto-Encoder (2018, background): “Several works (Chung et al., 2015; Lee et al., 2018; Archer et al., 2015; Fraccaro et al., 2016; Liu et al., 2017; Goyal et al., 2017; Buesing et al., 2018; Serban et al., 2017) fall into this framework.”
Linear dynamical neural population models through nonlinear embeddings (2016, methodology): “In our case, we use an approximate posterior that is designed explicitly to parameterize a temporally correlated approximate posterior [17].”
Probabilistic Recurrent State-Space Models (2018, unknown): “A variety of stochastic deep recurrent models has been presented based on Stochastic Gradient Variational Bayes (SGVB) (Bayer & Osendorfer, 2014; Krishnan et al., 2015; Watter et al., 2015; Chung et al., 2015; Archer et al., 2015; Karl et al., 2016; Fraccaro et al., 2016; Gemici et al., 2017).”
Gaussian variational approximation with sparse precision matrices (2016, methodology): “Archer et al. (2016) develop “black-box” variational inference (Ranganath et al., 2014) for SSMs, where a Gaussian variational approximation is considered for the latent states.”
Structured VAEs: Composing Probabilistic Graphical Models and Variational Autoencoders (2016, methodology): “…we focus on traditional probabilistic graphical models here, the SVAE generative model can also be an RNN expressing ﬂexible nonlinear dynamics (Archer et al., 2015), though such models would not in general admit the efﬁcient inference algorithms and natural gradients provided by exponential…”
Dynamical Variational Autoencoders: A Comprehensive Review (2020, background): “Black-box deep SSM: In a similar spirit, the authors of (Archer et al., 2015) also focus on the structure of the approximate posterior distribution to improve the computational efficiency of the inference.”
KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics (2021, methodology): “Variational inference [57]–[59] is commonly used in connection with SS models, as in [29]–[31], [33], [34], by casting the Bayesian inference task to optimization of a parameterized posterior and maximizing an objective.”