Physics-Informed Neural Operator for Learning Partial Differential Equations

By Zong-Yi Li, Hongkai Zheng, Nikola B. Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, Anima Anandkumar. ACM / IMS Journal of Data Science, 2021

856 citations recorded by the source corpus.

In this article, we propose physics-informed neural operators (PINO) that combine training data and physics constraints to learn the solution operator of a given family of parametric Partial Differential Equations (PDE). PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. Specifically, in PINO, we combine coarse-resolution training data with PDE constraints imposed at a higher resolution. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families and shows no degradation in accuracy even under zero-shot super-resolution, that is, being able to predict beyond the resolution of training data. PINO uses the Fourier neural operator (FNO) framework that is guaranteed to be a universal approximator for any continuous operator and discretization convergent in the limit of mesh refinement. By adding PDE constraints to FNO at a higher resolution, we obtain a high-fidelity reconstruction of the ground-truth operator. Moreover, PINO succeeds in settings where no training data is available and only PDE constraints are imposed, while previous approaches, such as the Physics-Informed Neural Network (PINN), fail due to optimization challenges, for example, in multi-scale dynamic systems such as Kolmogorov flows. Highlights PROBLEM STATEMENT Machine learning methods have recently shown promise in solving partial differential equations (PDEs) raised in science and engineering. They can be classified into two broad categories: approximating the solution function and learning the solution operator. The Physics-Informed Neural Network (PINN) is an example of the former while the Fourier neural operator (FNO) is an example of the latter. Both these approaches have shortcomings. The optimization in PINN is challenging and prone to failure, especially on multi-scale dynamic systems. FNO does not suffer from this optimization issue since it carries out supervised learning on a given dataset, but obtaining such data may be too expensive or infeasible. In this paper, we consider a new learning paradigm, aiming to overcome the optimization challenge in PINN and relieve the data requirement in FNO. METHODS In this paper, we propose physics-informed neural operators (PINO) that combine training data and physics constraints to learn the solution operator of a given family of parametric PDEs. In the operator-learning phase, PINO learns the solution operator over multiple instances of the parametric PDE family using training data and physics constraints. In the instance-wise fine-tuning phase, PINO optimizes the pre-trained operator ansatz for the querying instance of the PDE using the physics constraints only. Specifically, we combine coarse-resolution training data with PDE constraints imposed at a higher resolution. By adding PDE constraints to FNO at a higher resolution, we obtain a high-fidelity reconstruction of the ground-truth operator. RESULTS The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families and shows no degradation in accuracy even under zero-shot super-resolution, i.e., being able to predict beyond the resolution of training data. Experiments show PINO outperforms previous ML methods on many popular PDE families while retaining the extraordinary speed-up of FNO compared to solvers. With the equation constraints, PINO requires few to no data to learn the Burgers, Darcy, and Navier-Stokes equation. In particular, PINO accurately solves long temporal transient flows and Kolmogorov flows where other baseline methods fail to converge. SIGNIFICANCE PINO uses the neural operator framework that is guaranteed to be a universal approximator for any continuous operator and discretization convergent in the limit of mesh refinement. Moreover, PINO succeeds in settings where no training data is available and only PDE constraints are imposed. These advantages could lead to applications such as weather forecast, airfoil designs, and turbulence control.

Subfields: Aerospace Engineering, Artificial Intelligence, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/319d0aea3b8d5500ea01d722bf9deaf776915634

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, positive): “We will explore physics-informed approaches [54] and generate time-dependent high-fidelity simulations in the future.”
Neural operators for accelerating scientific simulations and design (2023, positive): “In contrast, as discussed in [18], band-limited operators and representation-equivalent operators [23], such as spectral neural operators [24], cannot generate new (higher) frequencies since their representation space is fixed.”
KAN: Kolmogorov-Arnold Networks (2024, positive): “We refer to Deep Ritz Method [79], PINNs [22, 23] for PDE solving, and Fourier Neural operator [80], PINOs [81, 82, 83], DeepONet [84] for operator learning methods learning the solution map.”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “In this section, we ﬁrst present the benchmark results on the standard neural operator benchmark problems from Li et al. (2021a) and compare our model with several other state-of-the-art operator learning frameworks, including Galerkin/Fourier Transformer (G.T. / F.T.) from Cao (2021) and…”
FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators (2022, methodology): “The Fourier Neural Operator has been extended to be physics-informed [Li et al., 2021b].”
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries (2022, background): “Furthermore, it is possible to extend the geo-FNO framework to the physics-informed neural operator setting which incorporates PDE constraints [17].”
Scientific discovery in the age of artificial intelligence (2023, result): “In contrast to numerical algebra solvers, AI-based neural solvers integrate data and physics more seamlessly 173,174 .”
FourCastNet: Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators (2022, methodology): “The Fourier Neural Operator (FNO) is a class of integral operators that efficiently implements global convolution via Fast Fourier Transform (FFT) which has been used very successfully to solve nonlinear and chaotic PDEs [20].”