Model Reduction and Neural Networks for Parametric PDEs

By Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, Andrew M. Stuart. SMAI Journal of Computational Mathematics, 2020

476 citations recorded by the source corpus.

We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. Numerically we demonstrate the effectiveness of the method on a class of parametric elliptic PDE problems, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare our method with existing algorithms from the literature.

Subfields: Computational Mechanics, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8bd148ef2082c022a60ee02d56b0e602136761ab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Transformer for Partial Differential Equations' Operator Learning (2022, background): “The promising features of data-driven operator learning, in particular the generalization capability within a family of PDE and the potential adaptivity to different discretizations, have given rise to a wide array of recent research works [5, 9, 10, 23, 32, 41, 42, 46, 53, 61, 86].”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, methodology): “There are also other methods for operator regression such as [28, 29, 30, 31], but in this work we only consider DeepONet and FNO.”
Multiwavelet-based Operator Learning for Differential Equations (2021, background): “Finally, the closest stream of work to the problem we investigate is represented by the “Neural Operators" [16, 49, 50, 51, 58].”
MIONet: Learning multiple-input operators via tensor product (2022, methodology): “In the experiments, we directly evaluate the function values at uniform grid points as the input of branch nets, i.e., each ϕ takes 100 equidistant sampling points in [0 , 1] for each input function.”
Factorized Fourier Neural Operators (2021, methodology): “Similarly, Bhattacharya et al. (2020) use PCA to map from an inﬁnite dimensional input space into a latent space, on which a neural network operates before being transformed to the output space.”
Message Passing Neural PDE Solvers (2022, methodology): “Inﬁnite-dimensional operator methods (Li et al., 2020c;a; Bhattacharya et al., 2021; Patel et al., 2021) by contrast resolve this issue.”
Fourier Neural Operator for Parametric Partial Differential Equations (2020, background): “Recently, a new line of work proposed learning mesh-free, inﬁnite-dimensional operators with neural networks (Lu et al., 2019; Bhattacharya et al., 2020; Nelsen & Stuart, 2020; Li et al., 2020b;a; Patel et al., 2021).”
Neural Operator: Graph Kernel Network for Partial Differential Equations (2020, methodology): “• PCA+NN is an instantiation of the methodology proposed in [Bhattacharya et al., 2020]: using PCA as an autoencoder on both the input and output data and interpolating the latent spaces with a neural network.”
Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “For ablations, we consider channel dimensions [32, 48, 64, 80], latent space [32, 48, 64, 80], and radius from 0.025 to 0.055 (with the domain size normalized to [-1, 1]).”
Choose a Transformer: Fourier or Galerkin (2021, background): “The huge success of many Galerkin-type methods in approximating solutions to operator equations such as PDEs [20] attributes partly to the following two fundamental properties of Hilbert spaces (see e.g., [21, Chapter 4]): for ( H , (cid:104)· , ·(cid:105) ) • Let Y be a convex and complete subset o”
POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition (2021, methodology): “Very recently, an ANN-based methodology is proposed to learn mappings between inﬁnite-dimensional spaces for parametric PDEs in [39], exploiting POD and showing mesh-independent properties.”