B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

By Liu Yang, Xuhui Meng, G. Em Karniadakis. 2020

1,163 citations recorded by the source corpus.

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINN posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loeve (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Subfields: Aerospace Engineering, Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “[300], where they use a Bayesian NN, which naturally encodes uncertainty, as a surrogate for a PDE solution.”
PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, background): “[38], where both the VI and Hamiltonian Monte Carlo formulations are compared.”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, methodology): “This has been studied by using the Bayesian framework [34].”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, methodology): “Noisy data are omnipresent and eﬀective NN training and inference should be stable to noise and provide answers with uncertainty quantiﬁcation using, e.g., Bayesian NN as in [37, 35, 38, 39].”
Deep-learning seismology (2022, methodology): “…(for example, simultaneously in architecture design and loss function), using physics-based GANs to enforce realism for data generation ( 151 ), constraining a model statistically ( 151 ), quantifying uncertainties ( 152 ), and learning physical operators ( 153 ) are other directions of interest.”
Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs (2020, background): “However, dealing with data, perturbed by larger amplitudes of noise, necessitates a Bayesian framework, such as the one considered in [38].”
When and why PINNs fail to train: A neural tangent kernel perspective (2020, background): “…[6, 7], meta-material design [8, 9, 10], free boundary problems [11], Bayesian networks and uncertainty quantiﬁcation [12, 13, 14, 15, 16], high-dimensional PDEs [17, 18], stochastic differential equations [19], fractional differential equations [20, 21], and beyond [22, 23, 24, 25].”
Physics‐Informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions (2021, methodology): “They provide a flexible framework to incorporate multiple data types and/or any apriori knowledge of the structure imaged and hence offer a great potential for joint and Bayesian inversions (Yang et al., 2021).”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, methodology): “For example, one extension is to include uncertainty estimation within PINNs, where Yang et al. [2021] proposed Bayesian PINNs which use Bayesian neural networks to estimate uncertainty in the PINN solution.”
Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks (2021, methodology): “A pressure reconstruction method, such as solving the Poisson equation, is required for pressure inference, which is used routinely in PIV (Wang et al. 2016; Zhang et al. 2020).”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “This is explored in Yang et al. [277], where they use a Bayesian NN, which naturally encodes uncertainty, as a surrogate for a PDE solution.”
On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks (2020, background): “PINNs in particular, have demonstrated remarkable power in applications including fluid dynamics [12, 13, 14], biomedical engineering [15, 16, 17], meta-material design [18, 19], free boundary problems [20], Bayesian networks and uncertainty quantification [21, 22], high dimensional PDEs [23, 24, 25”