Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
By Yinhao Zhu, Nicholas Zabaras, Phaedon-Stelios Koutsourelakis, Paris Perdikaris. 2019
1,046 citations recorded by the source corpus.
Abstract Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the governing equations of the physical model in the loss/likelihood functions. The resulting physics-constrained, deep learning models are trained without any labeled data (e.g. employing only input data) and provide comparable predictive responses with data-driven models while obeying the constraints of the problem at hand. This work employs a convolutional encoder-decoder neural network approach as well as a conditional flow-based generative model for the solution of PDEs, surrogate model construction, and uncertainty quantification tasks. The methodology is posed as a minimization problem of the reverse Kullback-Leibler (KL) divergence between the model predictive density and the reference conditional density, where the later is defined as the Boltzmann-Gibbs distribution at a given inverse temperature with the underlying potential relating to the PDE system of interest. The generalization capability of these models to out-of-distribution input is considered. Quantification and interpretation of the predictive uncertainty is provided for a number of problems.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability, AI for Physical Sciences
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- G. Em Karniadakis
- Paris Perdikaris
- Yibo Yang
- Maizar Raissi
- Xuhui Meng
- Ameya Dilip Jagtap
- Lu Lu
- Zhiping Mao
In Statistical and Nonlinear Physics
In Statistics and Probability
In AI for Physical Sciences
- G. Em Karniadakis
- Paris Perdikaris
- Maizar Raissi
- Yibo Yang
- Jian-Xun Wang
- Sifan Wang
- Lu Lu
- Ameya Dilip Jagtap
Citations in the corpus
- Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, neutral): “PINNs have been extended to convolutional [340] and graph neural network [270] backbones.”
- Efficient training of physics‐informed neural networks via importance sampling (2021, neutral): “, 2019), (4) uncertainty quantification (e.g., Meng & Karniadakis, 2019; Kissas et al., 2019; Nabian & Meidani, 2019; Raissi et al., 2018; Raissi, Wang, et al., 2019; Xu & Darve, 2019; Y. Yang & Perdikarisa, 2019; Zhu et al., 2019), (5) solving stochastic PDEs (e.”
- PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “5 (left panel), for which PDEconstrained learning is unable to be formulated using classic CNNs [3, 15, 67, 84].”
- Self-adaptive loss balanced Physics-informed neural networks (2021, background): “The quantification and interpretation of the predictive uncertainty are also provided [31].”
- DeepXDE: A Deep Learning Library for Solving Differential Equations (2019, background): “Among these approaches, some can only be applied to particular types of problems, such as image-like input domain [28, 33, 58] or parabolic PDEs [6, 19].”
- Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems (2021, background): “To reduce training costs and enable efficient learning, discrete PINNs that leverage convolution operations and numerical discretizations have begun to spur interests due to their better efficiency and scalability [19,20].”
- Physics-informed machine learning: case studies for weather and climate modelling (2021, methodology): “Probabilistic ML models are amenable for efficient UQ, as shown by Zhu et al. [16] in using conditional generative models to model uncertainties in solving PDEs.”
- Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data (2020, background): “Researchers have recently started to explore the uncertainty quantification (UQ) analysis of physics-constrained deep learning by using arbitrary polynomial chaos [31] and variational inference [27, 32, 33].”
- Deep learning in nano-photonics: inverse design and beyond (2020, methodology): “Therefore, replacing conventional physics simulations by surrogate ANNs is a natural solution to speed-up the inverse design of photonic nano-structures via global optimization heuristics [59,60].”
- Extended Physics-informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition based Deep Learning Framework for Nonlinear Partial Differential Equations (2020, background): “flows [23], turbulent fluid flows [26,27], quantification of surface breaking cracks [19], uncertainty quantification [15,16], elastodynamics [28] etc.”
- When and why PINNs fail to train: A neural tangent kernel perspective (2020, background): “However, PINNs using fully connected architectures often fail to achieve stable training and produce accurate predictions, especially when the underlying PDE solutions contain high-frequencies or multi-scale features [26, 13, 27].”
- Physics-informed deep learning for incompressible laminar flows (2020, background): “[12] proposed a physics-constrained convolutional encoder-decoder network and a generative model for modeling of stochastic fluid flows.”