Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations

By Maizar Raissi, Paris Perdikaris, G. Em Karniadakis. 2017

315 citations recorded by the source corpus.

We introduce the concept of numerical Gaussian processes, which we define as Gaussian processes with covariance functions resulting from temporal discretization of time-dependent partial differential equations. Numerical Gaussian processes, by construction, are designed to deal with cases where (a) all we observe are noisy data on black-box initial conditions, and (b) we are interested in quantifying the uncertainty associated with such noisy data in our solutions to time-dependent partial differential equations. Our method circumvents the need for spatial discretization of the differential operators by proper placement of Gaussian process priors. This is an attempt to construct structured and data-efficient learning machines, which are explicitly informed by the underlying physics that possibly generated the observed data. The effectiveness of the proposed approach is demonstrated through several benchmark problems involving linear and nonlinear time-dependent operators. In all examples, we are able to r...

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

In Numerical Methods & Scientific Computing

Citations in the corpus

Neural Ordinary Differential Equations (2018, neutral): “One can train feed-forward or recurrent neural networks to approximate a differential equation (Raissi and Karniadakis, 2018; Raissi et al., 2018a; Long et al., 2017), with applications such as ﬂ uid simulation (Wiewel et al., 2018).”
Neural-net-induced Gaussian process regression for function approximation and PDE solution (2018, methodology): “2 1D Burgers’ equation Consider the equation [7]”
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations (2018, background): “One could avoid the first drawback concerning numerical differentiation by assigning prior distributions in the forms of Gaussian processes [38, 39, 40, 41] or neural networks [35, 34] to the unknown solution u.”
Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems (2018, background): “…in the context of identifying nonlinear dynamics, there exist several deterministic and probabilistic tools including radial basis functions [6], neural networks [7], Gaussian processes [8, 9, 10, 11, 12], and nonlinear auto-regressive models such as NARMAX [13] and recurrent neural networks [14].”
Hidden physics models: Machine learning of nonlinear partial differential equations (2017, methodology): “This work should be considered a direct follow up on [1] in which a similar methodology was employed to infer solutions to time-dependent and non-linear partial diﬀerential equations, and eﬀectively quantify and propagate uncertainty due to noisy initial or boundary data.”
Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations (2017, background): “Through the lens of different benchmark problems, we highlighted the key features of physics informed neural networks in the context of data-driven solutions of partial differential equations [5, 6].”
Adversarial Uncertainty Quantification in Physics-Informed Neural Networks (2018, methodology): “In contrast to previous approaches to inferring solutions of partial differential equations from data [44, 45, 46, 47], the proposed methodology does not rely on Gaussian assumptions, and it can directly tackle nonlinear problems without any need for linearization.”
Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations (2017, background): “Although this important element was naturally addressed in previous work employing Gaussian processes [9], it not captured by the proposed methodology in its present form and requires further investigation.”
Machine learning of linear differential equations using Gaussian processes (2017, methodology): “Lastly, for general non-linear equations we can employ a sequential inference approach following the ideas recently put forth in [39].”
Adaptive activation functions accelerate convergence in deep and physics-informed neural networks (2019, methodology): “Further, their method was extended to nonlinear problems by Raissi and Karniadakis [23] and Raissi et al [24] in the context of solution inference and system identification.”
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations (2019, background): “The first problem is that of inference, filtering and smoothing, or data-driven solutions of partial differential equations [4,8] which states: given fixed model parameters λ what can be said about the unknown hidden state u(t, x) of the system? The second problem is that of learning, system identif”
Enforcing Constraints for Time Series Prediction in Supervised, Unsupervised and Reinforcement Learning (2019, unknown): “see [4, 20, 5, 13, 22, 7, 26, 16] and references therein.”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “Raissi et al [141, 142] used Gaussian process regression to construct representations of linear operator functionals, accurately inferring the solution and providing uncertainty estimates for a variety of physical problems; this was then extended in [140, 145].”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “To this end, recent advances include using Gaussian process regression [17, 18, 19, 20, 21, 22] and deep neural networks (DNNs) [23, 24, 25, 12, 26] to solve forward problems as well as Bayesian estimation [27] and DNNs [28, 29] for inverse problems.”
Machine Learning of Space-Fractional Differential Equations (2018, background): “13), and may be resolved by using higher-order differentiation with more time slices [25], as simply taking ∆t to be extremely small may cause the optimization to be dominated by numerical error in computing the kernels.”
Data-driven discovery of PDEs in complex datasets (2018, methodology): “Identification using Gaussian processes works by placing a Gaussian process prior on the unknown coefficients of the PDE and infer them by using maximum likelihood estimation [27, 30, 26].”
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems (2018, background): “For example, for the forward problem, some of the popular choices of machine learning tools are Gaussian process [3, 4, 5, 6, 7] and deep neural networks (DNNs) [8, 9, 10, 11, 12].”