Solving parametric PDE problems with artificial neural networks

By Yuehaw Khoo, Jianfeng Lu, Lexing Ying. European journal of applied mathematics, 2017

409 citations recorded by the source corpus.

The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modelled into the equations as random coefficients. However, very often the variability of physical quantities derived from PDE can be captured by a few features on the space of the coefficient fields. Based on such observation, we propose using neural network to parameterise the physical quantity of interest as a function of input coefficients. The representability of such quantity using a neural network can be justified by viewing the neural network as performing time evolution to find the solutions to the PDE. We further demonstrate the simplicity and accuracy of the approach through notable examples of PDEs in engineering and physics.

Subfields: Numerical Analysis, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c2bbc213e1fa9de078d189680345481302739fe2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Numerical Analysis

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations (2017, neutral): “…Bengio 2009; Cai 2017; Carleo and Troyer 2017; Chiaramonte and Kiener 2013; Dehghan et al. 2009; E et al. 2017c; Han and E 2016; Han et al. 2017; Khoo et al. 2017; Krizhevsky et al. 2012; Lagaris et al. 1998; LeCun et al. 1998, 2015; Lee and Kang 1990; Meade and Fernández 1994; Mehrkanoon and…”
Fourier Neural Operator for Parametric Partial Differential Equations (2020, background): “These approaches parameterize the solution operator as a deep convolutional neural network between ﬁnite-dimensional Euclidean spaces Guo et al. (2016); Zhu & Zabaras (2018); Adler & Oktem (2017); Bhatnagar et al. (2019); Khoo et al. (2017).”
DeepXDE: A Deep Learning Library for Solving Differential Equations (2019, background): “Among these approaches, some can only be applied to particular types of problems, such as image-like input domain [28, 33, 58] or parabolic PDEs [6, 19].”
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks (2019, background): “In application, how the F-Principle can help devise effective DNN-based approaches to solve scientiﬁc computing problems is an important issue (E et al., 2017; Khoo et al., 2017; He et al., 2018; Fan et al., 2018).”
Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks (2019, methodology): “, [3, 4, 8, 15, 34, 51, 52, 54, 58, 65, 80, 87, 92] for approximation methods for PDEs which are based on reformulations of PDEs as a deep learning problems.”
Message Passing Neural PDE Solvers (2022, methodology): “(4) Finite-dimensional operator methods (Raissi, 2018; Sirignano & Spiliopoulos, 2018; Bhatnagar et al., 2019; Guo et al., 2016; Zhu & Zabaras, 2018; Khoo et al., 2020), where dim(F) < ∞ are grid-dependent, so cannot generalize over geometry and sampling.”
Relu Deep Neural Networks and Linear Finite Elements (2018, background): “Recently, there are increased new research interests in the literature for the application of deep neural networks for numerical approximation of nonlinear and high dimensional PDEs as in [25, 24, 26].”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “The use of convolutional layers for structured grids (Khoo et al., 2021; Zhu & Zabaras, 2018; Bhatnagar et al., 2019; Pant et al., 2021), and graph layers for learning unstructured local relations in a system (de Avila Belbute-Peres et al., 2020; Iakovlev et al., 2021; Li & Farimani, 2022; Li et…”
Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification (2018, methodology): “[57] use a 3 layer convolutional DNN to learn a map between input coeﬃcients of SPDEs to a functional of the PDE solution.”
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems (2018, background): “For example, for the forward problem, some of the popular choices of machine learning tools are Gaussian process [3, 4, 5, 6, 7] and deep neural networks (DNNs) [8, 9, 10, 11, 12].”
A Multiscale Neural Network Based on Hierarchical Matrices (2018, methodology): “For general applications of machine learning to nonlinear numerical analysis problems the work of Raissi and Karnidiakis uses machine learning, in particular, Gaussian processes, to find parameters in nonlinear equations [41]; Khoo, Lu and Ying used neural network in the context of uncertainty quant”
Multiwavelet-based Operator Learning for Differential Equations (2021, background): “The FNO model with higher values of k m has better performance due to more Fourier bases for representing the high-frequency signal, while MWT does better even with low modes in its A, B, C CNNs, highlighting the importance of using wavelet-based ﬁlters in the signal processing.”
Data Driven Governing Equations Approximation Using Deep Neural Networks (2018, neutral): “([29, 11, 27, 25]), uncertainty quantification ([49, 54]), etc.”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “To this end, recent advances include using Gaussian process regression [17, 18, 19, 20, 21, 22] and deep neural networks (DNNs) [23, 24, 25, 12, 26] to solve forward problems as well as Bayesian estimation [27] and DNNs [28, 29] for inverse problems.”
Neural Operator: Learning Maps Between Function Spaces (2021, background): “Supervised learning has the potential to address these challenges when designed in a way that allows for the emulation of mappings between function spaces (Khoo et al., 2021; Lu et al., 2019; Bhattacharya et al., 2020; Nelsen and Stuart, 2021; Li et al., 2020a,b,c; Patel et al., 2021; Opschoor et al”
Deep Multiscale Model Learning (2018, methodology): “In [35], a neural network was proposed to learn the physical quantity of interest as a function of random input coefficients; the accuracy and efficiency of the approach for solving parametric PDE problems was shown.”