Black-Box Variational Inference for Stochastic Differential Equations

By Tom Ryder, Andrew Golightly, A. Mcgough, Dennis Prangle. 2018

74 citations recorded by the source corpus.

Parameter inference for stochastic differential equations is challenging due to the presence of a latent diffusion process. Working with an Euler-Maruyama discretisation for the diffusion, we use variational inference to jointly learn the parameters and the diffusion paths. We use a standard mean-field variational approximation of the parameter posterior, and introduce a recurrent neural network to approximate the posterior for the diffusion paths conditional on the parameters. This neural network learns how to provide Gaussian state transitions which bridge between observations in a very similar way to the conditioned diffusion process. The resulting black-box inference method can be applied to any SDE system with light tuning requirements. We illustrate the method on a Lotka-Volterra system and an epidemic model, producing accurate parameter estimates in a few hours.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability, Probabilistic & Bayesian Methods, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bernhard Schölkopf
2. Stefan Bauer
3. Michael W. Mahoney
4. David Kristjanson Duvenaud
5. Ricky T. Q. Chen
6. Michael A. Osborne
7. Francis R. Bach
8. Andreas Krause

In Statistical and Nonlinear Physics

1. Ricky T. Q. Chen
2. Bernhard Schölkopf
3. Michael W. Mahoney
4. David Kristjanson Duvenaud
5. Christopher van der Heide
6. Fred Roosta
7. Liam Hodgkinson
8. Xuechen Li

In Statistics and Probability

1. Stefan Bauer
2. Bernhard Schölkopf
3. David Kristjanson Duvenaud
4. Christopher van der Heide
5. Michael W. Mahoney
6. Michael A. Osborne
7. Andreas Krause
8. Maxim Raginsky

In Probabilistic & Bayesian Methods

1. David Kristjanson Duvenaud
2. Bernhard Schölkopf
3. Stefan Bauer
4. Xuechen Li
5. Ricky T. Q. Chen
6. Michael W. Mahoney
7. Ting-Kam Leonard Wong
8. Christopher van der Heide

In Numerical Methods & Scientific Computing

1. Michael W. Mahoney
2. Michael A. Osborne
3. Andreas Krause
4. Maxim Raginsky
5. David Kristjanson Duvenaud
6. Bernhard Schölkopf
7. Xuechen Li
8. Ting-Kam Leonard Wong

Citations in the corpus

• Neural Ordinary Differential Equations (2018, neutral): “Ryder et al. (2018) use stochastic variational inference to recover the solution of a given stochastic differential equation.”
• AReS and MaRS - Adversarial and MMD-Minimizing Regression for SDEs (2019, methodology): “, 2015) or using excessive amount of computational resources even for small scale systems and state-of-the-art implementation (Ryder et al., 2018).”
• Scalable Gradients for Stochastic Differential Equations (2020, background): “In the machine learning literature, Ryder et al. [72] perform variational inference over the state and parameters for Euler-discretized latent SDEs and optimize the model with regular backpropagation.”
• Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit (2019, methodology): “Generative models of this type have been used considered in earlier work, first by Movellan et al. [2002] as a noisy continuous-time counterpart of recurrent neural nets, and, more recently, by Archambeau et al. [2007], Hashimoto et al. [2016], and Ryder et al. [2018].”
• Stochastic Normalizing Flows (2020, background): “The most obvious approach is to directly backpropagate through a numerical integration scheme such as in Ryder et al. (2018), but this does not scale well in T .”