Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks

By Dongkun Zhang, Ling Guo, G. Em Karniadakis. 2019

294 citations recorded by the source corpus.

One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs). We address this problem by taking advantage of recent advances in scientific machine learning and the dynamically orthogonal (DO) and bi-orthogonal (BO) methods for representing stochastic processes. Specifically, we propose two new Physics-Informed Neural Networks (PINNs) for solving time-dependent SPDEs, namely the NN-DO/BO methods, which incorporate the DO/BO constraints into the loss function with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the proposed methods overcome some of the drawbacks of the original DO/BO methods: we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and computational complexity as for forward problems. We demonstrate the capability of the proposed methods via several numerical examples: (1) A linear stochastic advection equation with deterministic initial condition where the original DO/BO method would fail; (2) Long-time integration of the stochastic Burgers' equation with many eigenvalue crossings during the whole time evolution where the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward and the inverse problem, including noisy initial data, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs (2019, background): “Examples include the Navier-Stokes and the KdV equations [1], stochastic PDEs [13–16], and fractional PDEs [17].”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, result): “…ﬁrst in 2017 [7, 8] for forward, inverse and hybrid problems, and since then there have also been rapid developments in this area [9, 10, 11, 12, 13, 14, 15, 16, 17], although the achievements so far in scientiﬁc and engineering applications have been modest compared to applications in…”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, background): “, integro-differential equations [21], fractional equations [21], surfaces PDEs [22] and stochastic differential equations [23].”
Self-adaptive loss balanced Physics-informed neural networks (2021, methodology): “These discretization-based methods approximate the solution of PDEs through their values at a set of grid points distributed over the spatial and temporal domain.”
Physics-informed neural networks with hard constraints for inverse design (2021, methodology): “1C): In classical Fourier analysis, many basis functions may be required approximate an arbitrary periodic function with a good accuracy, but as demonstrated in [61], we can use as few as two terms { cos( ) and neural networks are universal approximators of nonlinear continuous functions.”
When and why PINNs fail to train: A neural tangent kernel perspective (2020, background): “differential equations [19], fractional differential equations [20, 21], and beyond [22, 23, 24, 25].”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “For instance, Zhang et al (2020) observed that when a variable’s measurement was missing, the PINN implementation was capable of accurately predicting that variable.”
Extended Physics-informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition based Deep Learning Framework for Nonlinear Partial Differential Equations (2020, background): “Thanks to their universal approximation properties, they can be exploited to construct alternative approaches for solving PDEs.”