Multipole Graph Neural Operator for Parametric Partial Differential Equations

By Zong-Yi Li, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, Anima Anandkumar. Neural Information Processing Systems, 2020

539 citations recorded by the source corpus.

One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks. Graph neural networks (GNNs) have gained popularity in this area since graphs offer a natural way of modeling particle interactions and provide a clear way of discretizing the continuum models. However, the graphs constructed for approximating such tasks usually ignore long-range interactions due to unfavorable scaling of the computational complexity with respect to the number of nodes. The errors due to these approximations scale with the discretization of the system, thereby not allowing for generalization under mesh-refinement. Inspired by the classical multipole methods, we propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Our multi-level formulation is equivalent to recursively adding inducing points to the kernel matrix, unifying GNNs with multi-resolution matrix factorization of the kernel. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/3c46fa25b0215678c62524a7a9a883bdc8a0c041

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

U-FNO - an enhanced Fourier neural operator based-deep learning model for multiphase flow (2021, background): “Recently, a novel approach, the neural operator, has been proposed that directly learns the infinite-dimensional-mapping from any functional parametric dependence to the solution [26, 27, 28, 29].”
Factorized Fourier Neural Operators (2021, methodology): “Original FNO and geo-FNO architectures Motivated by the kernel formulation of the solution to linear PDEs using Green’s functions, Li et al. (2020b; 2022) propose an iterative approach to map input function a to output function u , where ◦ indicates function composition, L is the number of…”
Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems (2021, methodology): “Excellent performance of the graph-based learning has been reported in modeling various computational mechanics problems through classic data-driven training [34–39].”
Fourier Neural Operator for Parametric Partial Differential Equations (2020, background): “The neural operator, proposed in (Li et al., 2020b), is formulated as an iterative architecture v 0 (cid:55)→ v 1 (cid:55)→ . . . (cid:55)→ v T where v j for j = 0 , 1 , . . . , T − 1 is a sequence of functions each taking values in d v .”
GRAND: Graph Neural Diffusion (2021, methodology): “These have been applied to problems where the PDE can be described on a graph (Li et al., 2020c).”
Choose a Transformer: Fourier or Galerkin (2021, methodology): “[56] further improves the kernel approach by exploiting the multilevel grid structure.”
Neural Operator: Learning Maps Between Function Spaces (2021, methodology): “This construction was originally proposed in Li et al. (2020c).”
Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “To overcome this, prior works propose using multi-pole methods or multi-level graphs [11, 12] to help with global connectivity.”
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021, unknown): “Neural operator methods [30, 31, 32] represent the solution map of parametric PDEs as an integral Hilbert-Schmidt operator, whose kernel is parametrized and learned from paired observations, either using local message passing on a graph-based discretization of the physical domain [30, 31], or using ”
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries (2022, methodology): “FNO implements a series of layers computing global convolution operators with the fast Fourier transform (FFT) followed by mixing weights in the frequency domain and inverse Fourier transform.”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “…capability within a family of PDE and the potential adaptivity to diﬀerent discretizations, have given rise to a wide array of recent research works (Li et al., 2020b; 2021a; Wen et al., 2022; Rahman et al., 2022; Cao, 2021; Kissas et al., 2022; Lu et al., 2022; Cai et al., 2021; Bhattacharya et…”
Multiwavelet-based Operator Learning for Differential Equations (2021, methodology): “Speciﬁcally, we consider the graph neural operator ( GNO ) [50], the multipole graph neural operator ( MGNO ) [51], the LNO which makes a low-rank ( r ) representation of the operator kernel K p x, y q (also similar to unstacked DeepONet [52]), and the Fourier neural operator ( FNO ) [49].”
Message Passing Neural PDE Solvers (2022, methodology): “The solver is a graph neural network (GNN) (Scarselli et al., 2009; Kipf & Welling, 2017; Defferrard et al., 2016; Gilmer et al., 2017; Battaglia et al., 2018 solvers (Li et al., 2020b; De Avila Belbute-Peres et al., 2020), but ours have several notable features.”