Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations

By Maizar Raissi. 2018

226 citations recorded by the source corpus.

Classical numerical methods for solving partial differential equations suffer from the curse dimensionality mainly due to their reliance on meticulously generated spatio-temporal grids. Inspired by modern deep learning based techniques for solving forward and inverse problems associated with partial differential equations, we circumvent the tyranny of numerical discretization by devising an algorithm that is scalable to high-dimensions. In particular, we approximate the unknown solution by a deep neural network which essentially enables us to benefit from the merits of automatic differentiation. To train the aforementioned neural network we leverage the well-known connection between high-dimensional partial differential equations and forward-backward stochastic differential equations. In fact, independent realizations of a standard Brownian motion will act as training data. We test the effectiveness of our approach for a couple of benchmark problems spanning a number of scientific domains including Black-Scholes-Barenblatt and Hamilton-Jacobi-Bellman equations, both in 100-dimensions.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks (2019, background): “This simple and easy to program algorithm has been shown to be successful for diverse problems in physics and fluid mechanics [5, 6, 7], especially for inverse problems and even for discovering hidden physics [8].”
Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise (2019, methodology): “…diﬀerential equation in the context of neural network has been studied before, focusing either on understanding how dropout shapes the loss landscape [12], or on using stochastic diﬀerential equation as a universal function approximation tool to learn the solution of high dimensional PDEs [13].”
Self-adaptive loss balanced Physics-informed neural networks (2021, methodology): “These discretization-based methods approximate the solution of PDEs through their values at a set of grid points distributed over the spatial and temporal domain.”
Efficient training of physics‐informed neural networks via importance sampling (2021, background): “, 2019), (5) solving stochastic PDEs (e.g., Beck et al., 2018; Raissi, 2018b; Weinan et al., 2017; L. Yang et al., 2018), and (6) physicsdriven regularization of neural network surrogates (e.”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “There have only been very few works published on data-driven methods for SDEs, e.g., [30, 31], for forward problems; for the inverse problem, Zhang et.al.”
On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks (2020, background): “We believe that this may be one of the fundamental reasons that cause failure of PINNs in learning high-frequency or multi-scale solutions of PDEs.”
Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism (2020, background): “As part of the burgeoning field of scientific machine learning [1], physics-informed neural networks (PINNs) have emerged recently as an alternative to traditional partial different equation (PDE) solvers [2, 3, 4, 5].”
Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data (2018, methodology): “This choice is motivated by modern techniques for solving forward and inverse problems associated with partial differential equations, where the unknown solution is approximated either by a neural network [4, 5, 15, 16] or a Gaussian process [17–23].”
A Multiscale Neural Network Based on Hierarchical Matrices (2018, methodology): “Deep neural networks have been recently used to solve PDEs [7, 8, 12, 16, 26, 40, 43, 54, 56] and classical inverse problems [3, 49].”
A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate (2021, methodology): “Berg and Nystr ¨o m [46] proposed a uniﬁed deep neural network approach to approximate solutions to PDEs and then used deep learning to discover PDEs hidden in complex data sets from measurement data [47].”
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems (2018, background): “There have only been very few works published on solving stochastic diﬀerential equations using DNNs, e.g., [21, 22], for forward problems.”