Probabilistic ODE Solvers with Runge-Kutta Means

By Michael Schober, David Kristjanson Duvenaud, Philipp Hennig. 2014

127 citations recorded by the source corpus.

Runge-Kutta methods are the classic family of solvers for ordinary differential equations (ODEs), and the basis for the state of the art. Like most numerical methods, they return point estimates. We construct a family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution. In contrast to prior work, we construct this family such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties. Remaining degrees of freedom not identified by the match to Runge-Kutta are chosen such that the posterior probability measure fits the observed structure of the ODE. Our results shed light on the structure of Runge-Kutta solvers from a new direction, provide a richer, probabilistic output, have low computational cost, and raise new research questions.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

In Numerical Methods & Scientific Computing

Citations in the corpus

Neural Ordinary Differential Equations (2018, neutral): “There is also significant work on connecting Gaussian Processes (GPs) and ODE solvers (Schober et al., 2014).”
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations (2017, background): “tensions that can accommodate parameter inference, inverse, and model discovery problems [24]; and incorporate probabilistic time integration schemes that allow for a natural quantification of uncertainty due to time-stepping errors [30].”
Message Passing Neural PDE Solvers (2022, background): “, 2015), as has been done for RK4 solvers (Schober et al., 2014).”
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees (2015, methodology): “1 Recent developments include probabilistic solvers for linear systems [14] and differential equations [5, 26].”
Control Functionals for Quasi-Monte Carlo Integration (2015, methodology): “The methods that we describe are immediately applicable in a range of applications including marginalisation of hyper-parameters in classification [10], probabilistic inference for differential equations [28, 5], computation of model evidence [21] and approximation of the partition function in socia”
Bayesian Solution Uncertainty Quantification for Differential Equations (2013, methodology): “…higher order numerical solvers (e.g. explicit fourth order Runge–Kutta) use such a carefully constructed weighted average of model interrogations, our approach also provides a way to interpret numerical method in the context of a general probabilistic framework (e.g., Schober et al., 2014).”
Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems (2016, background): “…include probabilistic solvers for linear systems [Hennig, 2015], differential equations [Calderhead et al., 2009, Dondelinger et al., 2013, Schober et al., 2014, Barber and Wang, 2014, Conrad et al., 2015, Macdonald et al., 2015, Owhadi, 2016, Kersting and Hennig, 2016] and probabilistic…”
Bayesian Probabilistic Numerical Methods (2017, methodology): “This well-known fact from Bayesian decision theory5 is interesting in light of recent research in constructing PNMs whose mean functions correspond to classical numerical methods (Schober et al., 2014; Hennig, 2015; Särkkä et al., 2016; Teymur et al., 2016; Schober et al., 2016).”