Neural Operator: Learning Maps Between Function Spaces

By Nikola B. Kovachki, Zong-Yi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M. Stuart, Anima Anandkumar. arXiv.org, 2021

645 citations recorded by the source corpus.

The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We formulate the neural operator as a composition of linear integral operators and nonlinear activation functions. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. The proposed neural operators are also discretization-invariant, i.e., they share the same model parameters among different discretization of the underlying function spaces. Furthermore, we introduce four classes of efficient parameterization, viz., graph neural operators, multi-pole graph neural operators, low-rank neural operators, and Fourier neural operators. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider standard PDEs such as the Burgers, Darcy subsurface flow, and the Navier-Stokes equations, and show that the proposed neural operators have superior performance compared to existing machine learning based methodologies, while being several orders of magnitude faster than conventional PDE solvers.

View Paper: https://www.semanticscholar.org/paper/aafa9bb3ba269fb0561c0b2165d27464840545d7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers (2021, background): “The self-attention mechanism can be written as a kernel integration (Tsai et al., 2019; Cao, 2021; Kovachki et al., 2021).”
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere (2023, background): “The above formulation admits efficient learning of non-local operators, which is a desireable property for PDE applications (Kovachki et al., 2021).”
PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (2022, methodology): “Methods include Physics-informed neural networks (PINNs) [47], Neural operators (NOs) [32, 26], treating ResNet as a PDE approximant [49], custom architectures for specific problems such as TFNet for turbulent fluid flows [61], and generic image-to-image regression models such as the U-Net [48].”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, methodology): “In addition to computational tests, we also perform a theoretical comparison of DeepONet versus FNO, following the published work on the theory of DeepONet in [2, 24, 25, 26], and on the more recent theory of FNO in [4, 27].”
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries (2022, methodology): “The Geo-PINO method will be an optimization based spectral method for general geometry.”
MIONet: Learning multiple-input operators via tensor product (2022, background): “We note that some theoretical work [17] allows that the input function to be a vector-valued function, i.”
FourCastNet: Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators (2022, background): “1) Operator learning and transformers: Neural operators learn mappings between infinite dimensional spaces and have demonstrated impressive results in creating surrogate models of classical problems in fluid dynamics governed by the Navier-Stokes equations [9].”
Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “Neural operators are discretization convergent and can approximate general operators [3].”
Transformer for Partial Differential Equations' Operator Learning (2022, background): “Kovachki et al. (2021) further provide a theoretical analysis of the approximation capacity of kernel-integral-based learning framework.”
Physics-Informed Neural Operator for Learning Partial Differential Equations (2021, unknown): “The work from [49] compares the polynomial chaos expansion ( PCE ), DeepONet, and FNO, and shows that DeepONet has a higher approximation accuracy over PCE.”
Scientific discovery in the age of artificial intelligence (2023, background): “Neural operators learn representations invariant to discretization by learning mappings between function spaces 127,128 .”