Fourier Neural Operator for Parametric Partial Differential Equations
By Zong-Yi Li, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, Anima Anandkumar. International Conference on Learning Representations, 2020
4,467 citations recorded by the source corpus.
The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers' equation, Darcy flow, and the Navier-Stokes equation (including the turbulent regime). Our Fourier neural operator shows state-of-the-art performance compared to existing neural network methodologies and it is up to three orders of magnitude faster compared to traditional PDE solvers.
Subfields: Atmospheric Science, Computational Mechanics, Statistical and Nonlinear Physics
View Paper: https://www.semanticscholar.org/paper/2f7dc1ee85e9f6a97810c66016e09ffeed684f03
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Atmospheric Science
- Anima Anandkumar
- Zong-Yi Li
- Shuhao Cao
- Lu Lu
- Xuhui Meng
- Nikola B. Kovachki
- Siddhartha Mishra
- Zhiping Mao
In Computational Mechanics
- Anima Anandkumar
- Max Welling
- Zong-Yi Li
- G. Em Karniadakis
- Shuhao Cao
- Lu Lu
- Zhiping Mao
- Daniel E. Worrall
In Statistical and Nonlinear Physics
- Anima Anandkumar
- Zong-Yi Li
- G. Em Karniadakis
- Lu Lu
- Shuhao Cao
- Nikola B. Kovachki
- Max Welling
- Zijie Li
Citations in the corpus
- Transolver: A Fast Transformer Solver for PDEs on General Geometries (2024, methodology): “Li et al. (2021) proposed Fourier neural operators by approximating integration with linear projection in the Fourier domain.”
- Message Passing Neural PDE Solvers (2022, methodology): “We showed that these tricks reduce error explosion observed in training autoregressive models, including a SOTA neural operator method (Li et al., 2020a).”
- Choose a Transformer: Fourier or Galerkin (2021, methodology): “[57] proposes a discretization-invariant operator learner to achieve a state-of-the-art performance in certain benchmark problems.”
- Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, background): “5) [226] [251] [301] [303] [70] [196] [228] [248] [318] [82] [102] [144] [292] [216] [195] [23] [90] [320] [154] [58] [236] [53] [70] [192] [252] [26] [61] [149] [88] [299] [216] [195] [225] [145] [198] [171] [44] [188]”
- Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021, neutral): “( 34 ) ds ─ with periodic boundary conditions where t ∈ (0,1), the viscosity is set to = 0.01, and the initial condition u ( x ) is generated from a GRF ∼ (0,25 2 (− + 5 2 I ) −4 ), satisfying the periodic boundary conditions.”
- GRAND: Graph Neural Diffusion (2021, methodology): “These have been applied to problems where the PDE can be described on a graph (Li et al., 2020c).”
- FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting (2022, background): “(2020), they often suffer from problems such as gradient vanishing/exploding Pascanu et al. (2013), which limits their practical performance. Following the success of Transformer Vaswani et al. (2017) in the NLP and CV communities Vaswani et al.”
- Multiwavelet-based Operator Learning for Differential Equations (2021, methodology): “The loss function is taken as relative L2 error [47].”
- Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “A method that stands out is FNO , which uses Fourier transform [6].”
- Neural Operator: Learning Maps Between Function Spaces (2021, methodology): “This construction was originally proposed in Li et al. (2020c).”
- Physics-informed machine learning: case studies for weather and climate modelling (2021, methodology): “Li et al. [50] developed a new neural operator in Fourier space, allowing for an expressive and efficient architecture that solved the Navier–Stokes equations up to three orders of magnitude faster compared to traditional PDE solvers.”
- Monarch: Expressive Structured Matrices for Efficient and Accurate Training (2022, methodology): “We follow the experimental setting in FNO (Li et al., 2020) and apply a Monarch–based neural operator to the task of solving the Navier–Stokes PDE.”
- U-FNO - an enhanced Fourier neural operator based-deep learning model for multiphase flow (2021, methodology): “However, one type of neural operator, the Fourier neural operator (FNO), alleviates this issue through the implementation of a Fast Fourier Transform [34].”
- On universal approximation and error bounds for Fourier Neural Operators (2021, background): “For both these PDEs, we prove rigorously that there exists a Ψ-FNO which can approximate the underlying nonlinear operators efficiently, as we can show that the size of the Ψ-FNO only needs to grow polynomially in terms of the error.”
- Global Filter Networks for Image Classification (2021, background): “Very recent works also try to leverage Fourier transform to develop deep learning models to solve partial differential equations [25] and NLP tasks [23].”
- PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (2022, methodology): “Fourier neural operator (FNO) FNO [32] belongs to the family of Neural Operators (NOs), designed to approximate the forward propagator of PDEs.”
- Hyena Hierarchy: Towards Larger Convolutional Language Models (2023, methodology): “• FNO: Filters parametrized explicitly in the frequency-domain (Li et al., 2020).”
- GNOT: A General Neural Operator Transformer for Operator Learning (2023, methodology): “However, many methods like FNO (Li et al., 2020) using Fast Fourier Transform (FFT) and U-Net (Ronneberger et al.”