Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks

By Nicholas Geneva, Nicholas Zabaras. 2019

349 citations recorded by the source corpus.

Abstract In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models can require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model non-linear dynamical systems without training data at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.

Subfields: Aerospace Engineering, Computational Mechanics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Understanding and mitigating gradient pathologies in physics-informed neural networks (2020, background): “Although such approaches appear seemingly straightforward and have yielded remarkable results across a range of problems in computational science and engineering [19, 20, 21, 22, 23, 24, 25, 26] , the effects of the regularization mechanism in equation 1 remain poorly understood, and in several case”
What Are Bayesian Neural Network Posteriors Really Like? (2021, methodology): “…2019; Izmailov et al., 2019; Daxberger et al., 2020), with applications ranging from astrophysics (Cranmer et al., 2021) to automatic diagnosis of Diabetic Retinopathy (Filos et al., 2019), click-through rate prediction in advertising (Liu et al., 2017) and fluid dynamics (Geneva & Zabaras, 2020).”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “5) [226] [251] [301] [303] [70] [196] [228] [248] [318] [82] [102] [144] [292] [216] [195] [23] [90] [320] [154] [58] [236] [53] [70] [192] [252] [26] [61] [149] [88] [299] [216] [195] [225] [145] [198] [171] [44] [188]”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “Geneva and Zabaras (2020) propose a deep auto-regressive dense encoderdecoder and the physics-constrained training algorithm for predicting transient PDEs. They extend this model to a Bayesian framework to quantify both epistemic and aleatoric uncertainty. Finally, Grubǐsić et al (2021) also used an”
Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems (2021, background): “For instance, Zhu et al. [19] developed a physics-constrained convolutional an encoder–decoder to solve high-dimensional elliptic PDEs, and Geneva et al. [21] further extended this framework to dynamic hyperbolic PDEs with parametric initial conditions.”
PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “5 (left panel), for which PDEconstrained learning is unable to be formulated using classic CNNs [3, 15, 67, 84].”
Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data (2020, background): “Researchers have recently started to explore the uncertainty quantiﬁcation (UQ) analysis of physics-constrained deep learning by using arbitrary polynomial chaos [31] and variational inference [27, 32, 33].”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “poses an encoder-decoder for predicting transient PDEs with governing PDE constraints, which was further extended to deep auto-regressive dense encoder-decoders with a Bayesian framework using stochastic weight averaging to quantify uncertainty in [Geneva and Zabaras, 2020].”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, methodology): “[23], [24] and [25] instead used ﬁnite difference ﬁlters to construct a differential equation residual in a PINN-like loss function when training CNNs and RNNs to approximate the solution of the differential equation on a mesh.”
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021, positive): “Recent efforts [44, 16, 45, 46, 47] attempt to address some of these challenges by designing appropriate architectures and loss functions for learning discretized operators (i.”