Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
By Maizar Raissi. 2018
907 citations recorded by the source corpus.
A long-standing problem at the interface of artificial intelligence and applied mathematics is to devise an algorithm capable of achieving human level or even superhuman proficiency in transforming observed data into predictive mathematical models of the physical world. In the current era of abundance of data and advanced machine learning capabilities, the natural question arises: How can we automatically uncover the underlying laws of physics from high-dimensional data generated from experiments? In this work, we put forth a deep learning approach for discovering nonlinear partial differential equations from scattered and potentially noisy observations in space and time. Specifically, we approximate the unknown solution as well as the nonlinear dynamics by two deep neural networks. The first network acts as a prior on the unknown solution and essentially enables us to avoid numerical differentiations which are inherently ill-conditioned and unstable. The second network represents the nonlinear dynamics and helps us distill the mechanisms that govern the evolution of a given spatiotemporal data-set. We test the effectiveness of our approach for several benchmark problems spanning a number of scientific domains and demonstrate how the proposed framework can help us accurately learn the underlying dynamics and forecast future states of the system. In particular, we study the Burgers', Korteweg-de Vries (KdV), Kuramoto-Sivashinsky, nonlinear Schr\"{o}dinger, and Navier-Stokes equations.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Numerical Methods & Scientific Computing, AI for Physical Sciences
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- G. Em Karniadakis
- Paris Perdikaris
- Yibo Yang
- Maizar Raissi
- Stefania Fresca
- Andrea Manzoni
- Yang Liu
- Jens Berg
In Computer Vision and Pattern Recognition
- G. Em Karniadakis
- Paris Perdikaris
- Sifan Wang
- Stefania Fresca
- Andrea Manzoni
- Yang Liu
- Yibo Yang
- Timon Rabczuk
In Statistical and Nonlinear Physics
- G. Em Karniadakis
- Paris Perdikaris
- Andrea Manzoni
- Sifan Wang
- Yibo Yang
- Stefania Fresca
- Maizar Raissi
- Timon Rabczuk
In Numerical Methods & Scientific Computing
- Maizar Raissi
- G. Em Karniadakis
- Paris Perdikaris
- Yibo Yang
- Stefania Fresca
- Andrea Manzoni
- Jens Berg
- Kaj Nyström
In AI for Physical Sciences
- G. Em Karniadakis
- Paris Perdikaris
- Maizar Raissi
- Stefania Fresca
- Andrea Manzoni
- Sifan Wang
- Yibo Yang
- Jens Berg
Citations in the corpus
- Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “…have been used in many scientific applications, including fluid mechanics modeling (Raissi et al., 2020), cardiac activation mapping (Sahli Costabal et al., 2020), stochastic systems modeling (Yang and Perdikaris, 2019), and discovery of differential equations (Raissi et al., 2018; Raissi, 2018).”
- Physics informed deep learning for computational elastodynamics without labeled data (2020, methodology): “More recently, Raissi et al. [9, 10, 15, 16] has inherited and extended this concept, leveraged the strong expressibility of DNNs, and developed the general PINN framework to solve the forward and inverse problems involving the system of nonlinear PDEs with small datasets or even without any…”
- Understanding and mitigating gradient pathologies in physics-informed neural networks (2020, background): “Although such approaches appear seemingly straightforward and have yielded remarkable results across a range of problems in computational science and engineering [19, 20, 21, 22, 23, 24, 25, 26] , the effects of the regularization mechanism in equation 1 remain poorly understood, and in several case”
- Differentiable Physics-informed Graph Networks (2019, background): “Based on this fundamental idea, a number of works for simulating and discovering PDEs have been published (Raissi & Karniadakis, 2018; Raissi, 2018).”
- Adversarial Uncertainty Quantification in Physics-Informed Neural Networks (2018, background): “[6, 7, 9, 10] have demonstrated how classical conservation laws and numerical discretization schemes can be used as structured prior information that can enhance the robustness and efficiency of modern machine learning algorithms, introducing a new class of data-driven solvers, as well as a physics-”
- CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method (2021, background): “Moreover, PINNs offer advantage of being seamlessly extended to tackle real world inverse problems [2,30].”
- When and why PINNs fail to train: A neural tangent kernel perspective (2020, background): “However, PINNs using fully connected architectures often fail to achieve stable training and produce accurate predictions, especially when the underlying PDE solutions contain high-frequencies or multi-scale features [26, 13, 27].”
- Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data (2018, methodology): “However, as we compute the derivatives required in equation (5), minimizing the loss function (6) might become a challenge [14], where the optimizer may fail to converge to the right values for the parameters of the neural networks.”
- A Comprehensive Deep Learning-Based Approach to Reduced Order Modeling of Nonlinear Time-Dependent Parametrized PDEs (2020, methodology): “, in the works by Karniadakis and coauthors [22, 23, 24, 25, 26]; rather, DL techniques are built upon it, to enhance the repeated evaluation of the FOM for different values of the parameters.”
- Identifying Causal Structure in Large-Scale Kinetic Systems (2018, background): “…are proposed in Todorovski and Dzeroski [1997], Zembowicz and Zytkow [1992], Washio et al. [1999], while modern approaches even try end-to-end learning [Raissi, 2018, Martius and Lampert, 2016] and combine the identification of underlying differential equations from data with prediction.”
- POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition (2021, methodology): “In this respect, DL does not replace the high-fidelity FOM as, e.g., in the works by Karniadakis and coauthors [19, 20, 21, 22, 43]; rather, DL techniques are built upon it, to enhance the repeated evaluation of the FOM for different values of the parameters.”
- Self-adaptive loss balanced Physics-informed neural networks (2021, background): “Recently, there is much effort to investigate physics-based models [15,16,17].”
- Potential, challenges and future directions for deep learning in prognostics and health management applications (2020, background): “…have been proposed for integrating domain knowledge in the learning process, including physics-informed neural networks (Raissi et al., 2019; Raissi, 2018), physics-guided neural networks (Karpatne et al., 2017), semantic-based regularization (Diligenti et al., 2017) and integration of…”
- Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, neutral): “Discover Governing Equations [30] [227] [37] [167] [168] [216] [123] [200][205] [52] [141] [290] [207] [155] [251]”
- Physics-informed neural network for lithium-ion battery degradation stable modeling and prognosis (2024, positive): “Inspired by 54,55 , we propose to use a more generalized function approximator g 0 ð(cid:2)Þ with parameters θ 0 to represent the non-linear dynamics g ( ⋅ ).”
- A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applications (2020, methodology): “…ML models trained using governing PDEs: Instead of solving PDEs using FE tools to generate training datasets, in several studies governing PDEs have been used directly to train ML models (Han et al., 2018; Raissi, 2018; Raissi et al., 2019, 2017a, 2017b; Sirignano and Spiliopoulos, 2018).”
- Message Passing Neural PDE Solvers (2022, neutral): “(4) Finite-dimensional operator methods (Raissi, 2018; Sirignano & Spiliopoulos, 2018; Bhatnagar et al., 2019; Guo et al., 2016; Zhu & Zabaras, 2018; Khoo et al., 2020), where dim( F ) < ∞ are grid-dependent, so cannot generalize over geometry and sampling.”