DGM: A deep learning algorithm for solving partial differential equations
By Justin A. Sirignano, Konstantinos Spiliopoulos. 2017
2,539 citations recorded by the source corpus.
High-dimensional PDEs have been a longstanding computational challenge. We propose a deep learning algorithm similar in spirit to Galerkin methods, using a deep neural network instead of linear combinations of basis functions. The PDE is approximated with a deep neural network, which is trained on random batches of spatial points to satisfy the differential operator and boundary conditions. The algorithm is mesh-less, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, sequences of spatial points are randomly sampled. We implement the approach for American options (a type of free-boundary PDE which is widely used in finance) in up to 100 dimensions. We call the algorithm a "Deep Galerkin Method (DGM)".
Subfields: Atmospheric Science, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, AI for Physical Sciences
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Atmospheric Science
- G. Em Karniadakis
- Jian-Xun Wang
- Luning Sun
- Anima Anandkumar
- Lu Lu
- Zong-Yi Li
- Yang Liu
- Paris Perdikaris
In Computer Vision and Pattern Recognition
In Statistical and Nonlinear Physics
- G. Em Karniadakis
- Jian-Xun Wang
- Han Gao
- Paris Perdikaris
- Luning Sun
- Lu Lu
- Anima Anandkumar
- Zong-Yi Li
In AI for Physical Sciences
Citations in the corpus
- Physics informed deep learning for computational elastodynamics without labeled data (2020, methodology): “This strategy has been widely used in existing PINN methods [8, 9, 19, 37].”
- Understanding and mitigating gradient pathologies in physics-informed neural networks (2020, background): “This is the main reason why the majority of recent works focused on applying deep learning tools to modeling and simulating physical systems governed by PDEs, still relies on generic fully connected architectures [12, 24, 23, 43, 26].”
- Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “(6) by minimizing the residual loss where the exact derivatives are calculated with automatic differentiation [32, 39, 33, 37].”
- Mean Field Analysis of Neural Networks: A Law of Large Numbers (2018, methodology): “Neural networks have also been used to solve high-dimensional PDEs in financial applications [45].”
- Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021, neutral): “More recently, ML tools are actively developed to infer solutions of PDEs [12, 13, 14, 15, 16, 17, 18], however most existing tools can only accommodate a fixed given set of input parameters or IBCs.”
- Enforcing Constraints for Time Series Prediction in Supervised, Unsupervised and Reinforcement Learning (2019, neutral): “see [4, 20, 5, 13, 22, 7, 26, 16] and references therein.”
- A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate (2021, background): “Sirignano and Spiliopoulos [45] gave a theoretic motivation of using deep neural networks as PDE approximators, which converges as the”
- Digital Twin: Values, Challenges and Enablers From a Modeling Perspective (2019, neutral): “[333] J.”
- Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “Additionally, the toolbox supports a wide range of NN, such as Fourier Networks and the DGM architecture, which is similar to the LSTM architecture.”
- A deep learning-based hybrid approach for the solution of multiphysics problems in electrosurgery (2019, background): “Recent works have attempted to build neural models that can make predictions about constitutive equations of materials [18], optimized numerical quadrature [19], solutions of partial differential equations [20,21], and fluid flows [15,22,23].”
- Message Passing Neural PDE Solvers (2022, neutral): “(4) Finite-dimensional operator methods (Raissi, 2018; Sirignano & Spiliopoulos, 2018; Bhatnagar et al., 2019; Guo et al., 2016; Zhu & Zabaras, 2018; Khoo et al., 2020), where dim( F ) < ∞ are grid-dependent, so cannot generalize over geometry and sampling.”
- PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “The effectiveness of the data-free FC-NN based PDE solution algorithm has been demonstrated on a number of canonical PDEs [41, 42] and stochastic PDEs [1, 43, 44].”
- A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applications (2020, methodology): “- Physics-informed ML models trained using governing PDEs: Instead of solving PDEs using FE tools to generate training datasets, in several studies governing PDEs have been used directly to train ML models (Han et al., 2018; Raissi, 2018; Raissi et al., 2019, 2017a, 2017b; Sirignano and Spiliopoulos”
- When and why PINNs fail to train: A neural tangent kernel perspective (2020, background): “…[6, 7], meta-material design [8, 9, 10], free boundary problems [11], Bayesian networks and uncertainty quantification [12, 13, 14, 15, 16], high-dimensional PDEs [17, 18], stochastic differential equations [19], fractional differential equations [20, 21], and beyond [22, 23, 24, 25].”
- Universal Differential Equations for Scientific Machine Learning (2020, methodology): “Given this difficulty, mesh-free methods based on universal approximators such as neural networks have been constructed to allow for direct solving of high dimensional PDEs [89, 90].”
- Characterizing possible failure modes in physics-informed neural networks (2021, background): “This has garnered interest and shown successful results in a wide variety of science and engineering problems and applications [3, 11, 16, 29–31 However, there have also been issues observed with this formulation.”
- Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning (2019, methodology): “E.g., Sirignano & Spiliopoulos (2017) proposed the Deep Galerkin method to solve a high-dimensional PDE from scattered data.”