Applied Stochastic Differential Equations

By Simo Särkkä, Arno Solin. 2019

553 citations recorded by the source corpus.

The topic of this book is stochastic differential equations (SDEs). As their name suggests, they really are differential equations that produce a different “answer” or solution trajectory each time they are solved. This peculiar behaviour gives them properties that are useful in modeling of uncertainties in a wide range of applications, but at the same time it complicates the rigorous mathematical treatment of SDEs. The emphasis of the book is on applied rather than theoretical aspects of SDEs and, therefore, we have chosen to structure the book in a way that we believe supports learning SDEs from an applied point of view. In the following, we briefly outline the purposes of each of the remaining chapters and explain how the chapters are connected to each other. In the chapters, we have attempted to provide a wide selection of examples of the practical application of theoretical and methodological results. Each chapter (except for the Introduction and Epilogue) also contains a representative set of analytic and hands-on exercises that can be used for testing and deepening understanding of the topics. Chapter 2 is a brief outline of concepts and solutions methods for deterministic ordinary differential equations (ODEs). We especially emphasize solution methods for linear ODEs, because the methods translate quite easily to SDEs. We also examine commonly used numerical methods such as the Euler method and Runge–Kutta methods, which we extend to SDEs in the later chapters. Chapter 3 starts with a number of motivating examples of SDEs found in physics, engineering, finance, and other applications. It turns out that in a modeling sense, SDEs can be regarded as noise-driven ODEs, but this notion should not be taken too far. The aim of the rest of the chapter is to show where things start to go wrong. Roughly speaking, with linear SDEs we are quite safe with this kind of thinking, but anything beyond them will not work.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Evaluation & Benchmarks

Citations in the corpus

Score-based Generative Modeling in Latent Space (2021, methodology): “The mean and variance are given by the differential equations and their solutions [104]: Here, z 0 denotes the mean of the distribution at t = 0 and σ 20 the component-wise variance at t = 0 .”
Score-Based Generative Modeling through Stochastic Differential Equations (2020, neutral): “(5.51) in S¨arkk¨a & Solin (2019) where Σ sub-VP p t q : “ Cov r x p t qs for a process t x p t qu 1 t “ 0 obtained by solving Eq.”
Gotta Go Fast When Generating Data with Score-Based Models (2021, background): “Thankfully, as long as the drift is afﬁne (i.e., f ( x , t ) = Ax + B ), the transition kernel p ( x ( t ) | x (0)) will always be normally distributed [Särkkä and Solin, 2019], which means that we can solve the forward diffusion in a single step.”
Solving Inverse Problems in Medical Imaging with Score-Based Generative Models (2021, background): “…the transition density p 0 t p x t | x 0 q is always a conditional linear Gaussian distribution, taking the form p 0 t p x t | x 0 q “ N p x t | α p t q x 0 , β 2 p t q I q where α : r 0 , 1 s Ñ R and β : r 0 , 1 s Ñ R can be derived analytically from f p t q and g p t q (Särkkä & Solin, 2019).”
Fast Sampling of Diffusion Models with Exponential Integrator (2022, background): “The forward diffusion of a DM for D -dimensional data is a linear diffusion described by the stochastic differential equation (SDE) (S¨arkk¨a & Solin, 2019) | ) is a simple Gaussian distribution, denoted as N ( µ t x 0 , Σ t ) , and the distribution π ( x T ) := p T ( x T ) is easy to sample from.”
Universal Speech Enhancement with Score-based Diffusion (2022, methodology): “Then, the associated transition kernel for the forward process (Särkkä & Solin, 2019) is since σ 0 → 0 (see also Song et al., 2021).”
Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, unknown): “(1) describes a Gaussian process, the mean and variance of the process state xt can be derived when its initial conditions are known [36].”
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2021, methodology): “…intuitively is also beneficial in our situation when using this diffusion for training SGMs. TheO term corresponds to an OrnsteinUhlenbeck process (Särkkä & Solin, 2019) in the velocity component, which injects noise such that the diffusion dynamics properly converge to equilibrium for any Γ>0.”
Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations (2022, background): “(6) Notably, we can easily sample from the transition distributions of each components, p0t(Xt|X0) and p0t(At|A0), as they are Gaussian distributions where the mean and variance are tractably determined by the coefficients of the forward diffusion process (Särkkä & Solin, 2019).”
GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series (2019, unknown): “In contrast to classical SDE inference (Särkkä & Solin, 2019), we consider the functions µ Y ( t ) and Σ Y ( t ) are parametrized by neural networks.”
Non-Denoising Forward-Time Diffusions (2023, methodology): “It is a deep result, and consequence of Doob h -transforms (Särkkä & Solin (2019, Chapter 7.9), Rogers & Williams (2000, Chapter IV.6.39)), that a diffusion processes pinned down on both ends is still a diffusion process.”
Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation (2020, background): “SDEs commonly arise in modeling random effects in systems with continuous state variables [23], or after diffusion approximation of Master equations for systems involving discrete state transitions in time [8,24,25].”
State Space Gaussian Processes with Non-Gaussian Likelihood (2018, background): “I Instead of working with the covariance function, the latent can be expressed in terms of an SDE I This is a continuous-time model [3], [6]: d f(t) d t = F f(t) + L w(t), f0 ∼ N (0,P∞)”