PDE-Net: Learning PDEs from Data
By Zichao Long, Yiping Lu, Xianzhong Ma, Bin Dong. 2017
886 citations recorded by the source corpus.
In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Numerical Methods & Scientific Computing, AI for Physical Sciences
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Anima Anandkumar
- G. Em Karniadakis
- Yoshua Bengio
- Pieter Abbeel
- Michael W. Mahoney
- Paris Perdikaris
- Quoc V. Le
In Statistical and Nonlinear Physics
- G. Em Karniadakis
- Lu Lu
- Anima Anandkumar
- Zong-Yi Li
- Xuhui Meng
- Zhiping Mao
- Paris Perdikaris
- Nikola B. Kovachki
In Numerical Methods & Scientific Computing
In AI for Physical Sciences
- G. Em Karniadakis
- Lu Lu
- Paris Perdikaris
- Xuhui Meng
- Zhiping Mao
- Nils Thürey
- Guofei Pang
- Maizar Raissi
Citations in the corpus
- Neural Ordinary Differential Equations (2018, neutral): “One can train feed-forward or recurrent neural networks to approximate a differential equation (Raissi and Karniadakis, 2018; Raissi et al., 2018a; Long et al., 2017), with applications such as fluid simulation (Wiewel et al.”
- Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow (2018, background): “nadversarial training approach to infer solutions for two-dimensional physics problems, such as heat diffusion and lid driven cavity flows. Other researchershaveproposedspecializednetworkstolearnPDEs[Long et al. 2017] by encoding the unknown differential operators with convolutions. While both works”
- Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, background): “Another approach is to embed physical laws into the learning systems, such as approximating differential operators with convolutions [24], enforcing hard constraint of mass conservation by learning the stream function [25] whose curl is guaranteed to be divergence-free.”
- fPINNs: Fractional Physics-Informed Neural Networks (2018, background): “In this paper, we focus on the identification of the parameters of fractional PDEs whose overall form is known but some coefficients and most importantly the operators themselves are not known.”
- Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals (2018, background): “16), unlike for rate constraints. Another type of constrained machine learning aims to constrain the model parameters to obey other properties, such as physical limits on the learned system (see e.g. Long et al. (2018); Stewart and Ermon (2017)). These constraints generally do not take the form of r”
- PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network (2018, methodology): “The proposed deep neural network is an upgraded version of our original PDE-Net [1].”
- Neural Jump Stochastic Differential Equations (2019, background): “More broadly, learning parameters of differential equations from data has been successful for physics-based problems with deterministic dynamics [33, 34, 35, 36, 37].”
- DeepXDE: A Deep Learning Library for Solving Differential Equations (2019, background): “Among these approaches, some can only be applied to particular types of problems, such as image-like input domain [28, 33, 58] or parabolic PDEs [6, 19].”
- Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics (2020, methodology): “For the simulated dynamics, we use an equation slightly modified based on the one in Long et al. (2018). where the index i points the i -th node whose coordinate is ( x i , y i ) in the 2D space ( [0 , 2 π ] × [0 , 2 π ] ) and ˆ x and ˆ y indicate x - and y -direction in the space. a ( i ) = 0 .”
- Solving parametric PDE problems with artificial neural networks (2017, methodology): “Thus our construction of the neural network in the proof is also related to the work [12] where multiple layers of convolutional NN are used to learn and solve evolutionary PDEs.”
- Machine Learning of Space-Fractional Differential Equations (2018, neutral): “See also [15] for a different approach.”
- Learning to Optimize: A Primer and A Benchmark (2021, methodology): “Beyond first-order algorithms, unrolling is also applied to second-order (Newton or Quasi-Newton) methods [236; 120], Differential Equations [131; 132; 259] and Interior Point method [58; 23], and Frank-Wolfe algorithm [125; 168].”
- Machine Learning in Seismology: Turning Data into Insights (2018, methodology): “Here, ML approaches developed to infer the governing partial differential equations directly from observations (Rudy et al. , 2017; Long et al. , 2018; Raissi and Karnia-dakis, 2018) may play an essential role in resolving the nature and relative contributions of the mechanisms responsible for…”
- Graph Element Networks: adaptive, structured computation and memory (2019, methodology): “Although GENs were originally inspired by finite element methods that model PDEs, we show that they are much more widely applicable.”
- Learning to Control PDEs with Differentiable Physics (2020, methodology): “In this context, several works have proposed methods for improving the solution of PDE problems (Long et al., 2018; Bar-Sinai et al., 2019; Hsieh et al., 2019) or used PDE formulations for unsupervised optimization (Raissi et al., 2018).”
- Efficient training of physics‐informed neural networks via importance sampling (2021, background): “…described by time-dependent nonlinear PDEs [2, 7], (2) solving variants of nonlinear PDEs [8–13], (3) data-driven discovery of PDEs (e.g. [2, 14–16]), (4) uncertainty quantification (e.g. [17–24]), (5) solving stochastic PDEs (e.g. [25–28]), and (6) physics-driven regularization of neural…”
- Characterizing possible failure modes in physics-informed neural networks (2021, background): “ML approaches for PDE problems have been increasing rapidly in recent years [13, 19].”
- Data Driven Governing Equations Approximation Using Deep Neural Networks (2018, background): “The topic has been explored in a series of recent articles, in the context of ODEs [36, 39]) and PDEs ([34, 33, 27]).”