Neural Operator: Graph Kernel Network for Partial Differential Equations

By Zong-Yi Li, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, Anima Anandkumar. International Conference on Learning Representations, 2020

1,099 citations recorded by the source corpus.

The classical development of neural networks has been primarily for mappings between a finite-dimensional Euclidean space and a set of classes, or between two finite-dimensional Euclidean spaces. The purpose of this work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators). The key innovation in our work is that a single set of network parameters, within a carefully designed network architecture, may be used to describe mappings between infinite-dimensional spaces and between different finite-dimensional approximations of those spaces. We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. The kernel integration is computed by message passing on graph networks. This approach has substantial practical consequences which we will illustrate in the context of mappings between input data to partial differential equations (PDEs) and their solutions. In this context, such learned networks can generalize among different approximation methods for the PDE (such as finite difference or finite element methods) and among approximations corresponding to different underlying levels of resolution and discretization. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/d407abcdf8b5ce6dbf9b9ab56357f4673676f951

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Factorized Fourier Neural Operators (2021, methodology): “Original FNO and geo-FNO architectures Motivated by the kernel formulation of the solution to linear PDEs using Green’s functions, Li et al. (2020b; 2022) propose an iterative approach to map input function a to output function u , where ◦ indicates function composition, L is the number of…”
Message Passing Neural PDE Solvers (2022, methodology): “Inﬁnite-dimensional operator methods (Li et al., 2020c;a; Bhattacharya et al., 2021; Patel et al., 2021) by contrast resolve this issue.”
Neural Operator: Learning Maps Between Function Spaces (2021, methodology): “This construction was originally proposed in Li et al. (2020c).”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, background): “DeepONet can be applied to partial diﬀerential equations (PDEs) but also to learning explicit mathematical operators, e.g., integration, fractional derivatives, Laplace transforms, etc. [20].”
Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “To efficiently compute the integral in equation (5), we truncate the integral to a local ball at x with radius r > 0 , as done in [2], We discretize the space and use a Riemann sum to compute the integral.”
MIONet: Learning multiple-input operators via tensor product (2022, background): “Several approaches of neural operators have been recently proposed such as deep operator network (DeepONet) [26, 27, 28] and Fourier neural operator (FNO) [20, 28], graph kernel network [21, 46], and others [33, 1, 41, 36].”
Neural operators for accelerating scientific simulations and design (2023, neutral): “On the Darcy equation, we train each fixed architecture UNet, FNO, and Graph neural operator (GNO) at a given resolution and test at that same resolution [11] (no super-resolution).”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “…the learned operators, despite showing promising performance on solving PDEs, are limited to have the same discretizations for the input and output (Anandkumar et al., 2020; Li et al., 2020b), ﬁxed input grids (Lu et al., 2021a) or to have equi-spaced grid points (Li et al., 2021a; Gupta et al.,…”
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021, positive): “Neural operator methods ( 32 – 34 ) represent the solution map of parametric PDEs as an integral Hilbert-Schmidt operator, whose kernel is parametrized and learned from paired observations, either using local message passing on a graph-based discretization of the physical domain ( 32 , 33 ) or…”
Fourier Neural Operator for Parametric Partial Differential Equations (2020, background): “The neural operator, proposed in (Li et al., 2020b), is formulated as an iterative architecture v 0 (cid:55)→ v 1 (cid:55)→ . . . (cid:55)→ v T where v j for j = 0 , 1 , . . . , T − 1 is a sequence of functions each taking values in d v .”
U-FNO - an enhanced Fourier neural operator based-deep learning model for multiphase flow (2021, background): “, 2020 [32]; and FNO is the original Fourier neural operator [34].”
Physics-informed machine learning: case studies for weather and climate modelling (2021, background): “Li et al. [58] developed graph NNs that capture long-range interactions in NN solutions to PDEs.”
Choose a Transformer: Fourier or Galerkin (2021, methodology): “The attentionbased operator learner has capacity to unearth structurally how this inverse operator’s responses on a subset, with various benefits articulated in [47, 49, 48, 54, 9].”
Physics-Informed Neural Operator for Learning Partial Differential Equations (2021, positive): “PINO requires fewer to no training data and generalizes better compared to the data-driven FNO [31], especially on high-resolution test instances.”
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries (2022, methodology): “Fourier transforms conducted in the computational domain are standard, since we have a uniform structured mesh in the computational domain.”
Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020, methodology): “Recently, GNNs have been directly used to learn solutions to PDEs by constructing graphs on the physical domain [4], and it is was further shown that GNNs can learn mesh-invariant solution operators [28].”