A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

By Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, G. Em Karniadakis. Computer Methods in Applied Mechanics and Engineering, 2021

711 citations recorded by the source corpus.

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019, and the second one, named Fourier Neural Operator or FNO, was published in 2020. In order to compare FNO with DeepONet for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also endow DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO. We consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries and especially noisy data, the performance of FNO deteriorates greatly. For example, for the instability wave analysis with only 0.1% noise added to the input data, the error of FNO increases 10000 times making it inappropriate for such important applications, while there is hardly any effect of such noise on the DeepONet. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

Subfields: Computational Mechanics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/93c9ea575192ce229900d9320623edbd647cdd63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Neural Operator: Learning Maps Between Function Spaces (2021, methodology): “DeepONet Recently, a novel operator regression architecture, named DeepONet, was proposed by Lu et al. (2019, 2021) that designs a generic neural network based on the approximation theorem from Chen and Chen (1995).”
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries (2022, methodology): “When applying it to irregular domain shapes, previous works usually embed the domain into a larger rectangular domain [16].”
MIONet: Learning multiple-input operators via tensor product (2022, methodology): “All the techniques developed for DeepONet in [28] can be directly used for MIONet.”
Neural operators for accelerating scientific simulations and design (2023, neutral): “While the original DeepONet model is constrained to be on a fixed input grid, later extensions of DeepOnet [32, 33] remove this limitation, which preserves discretization convergence, and thus is a special case of neural operators.”
Factorized Fourier Neural Operators (2021, background): “Lu et al. (2022) ﬁnd that the performance of the FNO deteriorates signiﬁcantly on complex geometries and noisy data.”
Transformer for Partial Differential Equations' Operator Learning (2022, background): “The promising features of data-driven operator learning, in particular the generalization capability within a family of PDE and the potential adaptivity to different discretizations, have given rise to a wide array of recent research works [5, 9, 10, 23, 32, 41, 42, 46, 53, 61, 86].”
Physics-Informed Neural Operator for Learning Partial Differential Equations (2021, neutral): “Similarly, Lu et al. [58] apply FNO with extension and interpolation on diverse geometries on the Darcy equation.”