Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems

By Dongkun Zhang, Lu Lu, Ling Guo, G. Em Karniadakis. 2018

528 citations recorded by the source corpus.

Abstract Physics-informed neural networks (PINNs) have recently emerged as an alternative way of numerically solving partial differential equations (PDEs) without the need of building elaborate grids, instead, using a straightforward implementation. In particular, in addition to the deep neural network (DNN) for the solution, an auxiliary DNN is considered that represents the residual of the PDE. The residual is then combined with the mismatch in the given data of the solution in order to formulate the loss function. This framework is effective but is lacking uncertainty quantification of the solution due to the inherent randomness in the data or due to the approximation limitations of the DNN architecture. Here, we propose a new method with the objective of endowing the DNN with uncertainty quantification for both sources of uncertainty, i.e., the parametric uncertainty and the approximation uncertainty. We first account for the parametric uncertainty when the parameter in the differential equation is represented as a stochastic process. Multiple DNNs are designed to learn the modal functions of the arbitrary polynomial chaos (aPC) expansion of its solution by using stochastic data from sparse sensors. We can then make predictions from new sensor measurements very efficiently with the trained DNNs. Moreover, we employ dropout to quantify the uncertainty of DNNs in approximating the modal functions. We then design an active learning strategy based on the dropout uncertainty to place new sensors in the domain in order to improve the predictions of DNNs. Several numerical tests are conducted for both the forward and the inverse problems to demonstrate the effectiveness of PINNs combined with uncertainty quantification. This NN-aPC new paradigm of physics-informed deep learning with uncertainty quantification can be readily applied to other types of stochastic PDEs in multi-dimensions.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Physics informed deep learning for computational elastodynamics without labeled data (2020, methodology): “…learning performance, the PINN framework has also been extended via incorporating variational/energy formulations of the residual physics loss function [37–40], distributed learning using domain decomposition [41, 42], and uncertainty quantiﬁcation via variational/Bayesian inference [26, 43–45].”
fPINNs: Fractional Physics-Informed Neural Networks (2018, methodology): “In this paper, we focus on the identification of the parameters of fractional PDEs whose overall form is known but some coefficients and most importantly the operators themselves are not known.”
B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data (2020, methodology): “In addition, we note that a non-Bayesian framework model, i.e., the dropout, has been used to quantify the uncertainty in deep neural networks, including the PINNs for solving PDEs [9, 19].”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, methodology): “In other applications, PINNs have been extended to fractional differential equations [28], stochastic differential equations [29, 30], and have even been used to discover underlying differential equations themselves [31].”
The Role of Machine Learning in the Understanding and Design of Materials (2020, methodology): “Several methods have been proposed for quantifying uncertainty, 51,157,158 such as measuring the distance of a new sample to the training data or using ensemble models.”
MIONet: Learning multiple-input operators via tensor product (2022, methodology): “As an active area of research in SciML, different methods have been developed to solve ordinary and partial differential equations (ODEs and PDEs) by parameterizing the solutions via neural networks (NNs), such as physics-informed NNs (PINNs) [48, 30, 35, 37, 47], deep Ritz method [43], and deep Gal”
A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks (2022, background): “In addition to standard PDEs, PINNs have also been extended to solve other types of PDEs, including integro-diﬀerential equations [3], fractional PDEs [9], and stochastic PDEs [10].”
Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks (2019, background): “In [9] we addressed the issue of total uncertainty for ﬁrst time and combined dropout and arbitrary polynomial chaos to model stochasticity in steady SPDEs.”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, background): “Another important extension is the uncertainty quantification for the PDE solutions inferred by neural networks [29,30,31,32,33].”
PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “The total uncertainty quantification (UQ) in PINN was also investigated based on adversarial inference [39] and arbitrary polynomial chaos in conjunction with dropout [40].”
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data (2019, methodology): “The PINN approach has been recently extended to assimilate multi-ﬁdelity training data [41], and its UQ analyses have been explored based on arbitrary polynomial chaos [42] and adversarial inference [43].”
DeepXDE: A Deep Learning Library for Solving Differential Equations (2019, background): “In addition, PINNs have been further extended to solve integro-differential equations (IDEs), fractional differential equations (FDEs) [42], and stochastic differential equations (SDEs) [57, 55, 41, 56].”
Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data (2020, background): “Researchers have recently started to explore the uncertainty quantiﬁcation (UQ) analysis of physics-constrained deep learning by using arbitrary polynomial chaos [31] and variational inference [27, 32, 33].”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “As a pedagogical example, in this paper we consider the one-dimensional stochastic elliptic equation, which retains most of the main features of more complex SDEs: where k ( x ; ω ) and f ( x ; ω ) are independent stochastic processes, and k ( x ; ω ) is strictly positive.”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, result): “…introduced ﬁrst in 2017 [7, 8] for forward, inverse and hybrid problems, and since then there have also been rapid developments in this area [9, 10, 11, 12, 13, 14, 15, 16, 17], although the achievements so far in scientiﬁc and engineering applications have been modest compared to applications in…”