A unified deep artificial neural network approach to partial differential equations in complex geometries

By Jens Berg, Kaj Nyström. 2017

683 citations recorded by the source corpus.

Abstract In this paper, we use deep feedforward artificial neural networks to approximate solutions to partial differential equations in complex geometries. We show how to modify the backpropagation algorithm to compute the partial derivatives of the network output with respect to the space variables which is needed to approximate the differential operator. The method is based on an ansatz for the solution which requires nothing but feedforward neural networks and an unconstrained gradient based optimization method such as gradient descent or a quasi-Newton method. We show an example where classical mesh based methods cannot be used and neural networks can be seen as an attractive alternative. Finally, we highlight the benefits of deep compared to shallow neural networks and device some other convergence enhancing techniques.

Subfields: Computational Mechanics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, background): “analytical and meshfree [33, 34]; (2) the loss function can be derived from the variational form [35, 36]; (3) stochastic gradient descent is used to train the network by randomly sampling mini-batches of inputs (spatial locations and/or time instances) [37, 35]; (4) deeper networks are used to brea”
Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks (2019, background): “[14] and more recently Berg and Nyström [16] showed fully connected networks can be used to learn PDE solutions on even complex domains.”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, methodology): “This formulation has since been extended to irregular domains [57, 58], and general schemes for constructing suitable ansatze have been proposed [59].”
CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method (2021, background): “Different architectures, from fully connected neural networks [2,3,9–12] to convolutional neural networks [13– 16], to recurrent neural networks [17–20], and generative adversarial neural networks [21,22], have been explored in the context of PINN.”
Geometric Deep Learning for Volumetric Computational Fluid Dynamics (2019, background): “Most of the research done consists in searching for a continuous approximation of the PDE, and often it constist in finding the approximating function given a discrete set of point in which we know the solution [32] ,[33] ,[34], [35],[36].”
Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks (2018, methodology): “In fact, these techniques have already been explored in the context of free energy calculations [1], partial differential equations [2, 3], and forcefield parameterization [4].”
Artificial Neural Network Methods for the Solution of Second Order Boundary Value Problems (2019, background): “…appears to produce reasonably accurate results, particularly for high-dimensional domains [Han, Jentzen and Weinan (2018)] and domains with complex geometries [Berg and Nyström (2018)], where the meshfree character of these methods makes them competitive with established discretization methods.”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “In Dwivedi and Srinivasan (2020) the authors address different 1D-2D linear advection and/or diffusion steady-state problems from Berg and Nyström (2018), by using their PIELM, a PINN combined with ELM (Extreme Learning Machine). A critical point is that the proposed PIELM only takes into account li”
Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks (2021, methodology): “[4, 5] and more recently Sheng and Yang [61] introduced a length factor (measure of the distance to the boundary) associated with the boundary to impose boundary conditions, and Berg and Nyström [7] approximated the distance function using a low-capacity neural network to impose boundary conditions ”
Efficient training of physics‐informed neural networks via importance sampling (2021, background): “hard assignment of constraints [1], and the unified approach [8]), but are not discussed here for brevity.”
An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications (2019, background): “The solution of φ is chosen in such a way that it satisfies all the boundary conditions as in [24, 25].”
hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition (2020, methodology): “The collocation formulation is used in [1] and PINNs [3].”
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data (2019, background): “In addition, contrary to the previous data-driven surrogates, the training of our DNN is solely driven by minimizing the residuals of the governing PDEs (i.e., conservation laws), where no expensive CFD simulation data is used.”