Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

By Liu Yang, Dongkun Zhang, G. Em Karniadakis. 2018

490 citations recorded by the source corpus.

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even for a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. We used three generators for the PI-GANs, two of them were feed forward deep neural networks (DNNs) while the other one was the neural network induced by the SDE. Depending on the data, we employed one or multiple feed forward DNNs as the discriminators in PI-GANs. Here, we have demonstrated the accuracy and effectiveness of PI-GANs in solving SDEs for up to 30 dimensions, but in principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

Subfields: Computational Mechanics, Statistical and Nonlinear Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems (2019, background): “Of course, nowadays many adversarial models share the same motivation and other stable formulations exist (e.g. Wasserstein GANs [59,60]).”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “5) [226] [251] [301] [303] [70] [196] [228] [248] [318] [82] [102] [144] [292] [216] [195] [23] [90] [320] [154] [58] [236] [53] [70] [192] [252] [26] [61] [149] [88] [299] [216] [195] [225] [145] [198] [171] [44] [188]”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “Work by Yang et al. [279] shows GANs with loss functions based on PDEs can be used to solve stochastic elliptic PDEs in up to 30 dimensions.”
SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics (2020, unknown): “…approaches [2, 3], dynamic mode decomposition (DMD) [4, 5] and Koopman theory [6–9], nonlinear autoregressive models [10, 11], neural networks [12–20], Gaussian process regression [21, 22], nonlinear Laplacian spectral analysis [23], diffusion maps [24], genetic programming [25–27], and…”
CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method (2021, background): “Different architectures, from fully connected neural networks [2,3,9–12] to convolutional neural networks [13– 15], to recurrent neural networks [16–19], and generative adversarial neural networks [20,21], have been explored in the context of PINN.”
AReS and MaRS - Adversarial and MMD-Minimizing Regression for SDEs (2019, methodology): “As recently shown by Yang et al. (2018), GANs have a huge potential for solving stochastic partial differential equations (SPDEs).”
PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “The effectiveness of the data-free FC-NN based PDE solution algorithm has been demonstrated on a number of canonical PDEs [41, 42] and stochastic PDEs [1, 43, 44].”
Deep Learning Techniques for Inverse Problems in Imaging (2020, methodology): “Deep learning techniques have been recently proposed to tackle these problems [41, 42], including methods that rely on generative models [43] constrained by partial differential equations; see also [44].”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “In particular, Yang et al [189] investigates the following, time independent, SDE”
Physics-informed machine learning: case studies for weather and climate modelling (2021, background): “[111] encode the governing physical laws in the form of stochastic differential equations into the architecture of GANs, and Stinis et al.”
hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition (2020, methodology): “…[22], several solvers have been developed, such as deep Galerkin method (DGM) [23] based on the least squares, physics-informed neural networks (PINNs) [3, 24] based on the collocation methods, and variational physics-informed neural networks (VPINNs) [25, 26] based on the Galerkin method.”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, methodology): “In other applications, PINNs have been extended to fractional differential equations [28], stochastic differential equations [29, 30], and have even been used to discover underlying differential equations themselves [31].”
Multiscale modeling meets machine learning: What can we learn? (2019, background): “Generative adversarial networks can also be useful for effective modeling of parameterized partial differential equations with thousands of uncertain parameters [140, 141].”