Machine learning of linear differential equations using Gaussian processes

By Maizar Raissi, Paris Perdikaris, G. Em Karniadakis. 2017

651 citations recorded by the source corpus.

This work leverages recent advances in probabilistic machine learning to discover governing equations expressed by parametric linear operators. Such equations involve, but are not limited to, ordinary and partial differential, integro-differential, and fractional order operators. Here, Gaussian process priors are modified according to the particular form of such operators and are employed to infer parameters of the linear equations from scarce and possibly noisy observations. Such observations may come from experiments or black-box computer simulations, as demonstrated in several synthetic examples and a realistic application in functional genomics. Employ probabilistic machine learning to discover governing equations expressed by parametric linear operators.Proper placement of Gaussian process priors allows one to efficiently infer model parameters via maximum likelihood estimation.A general treatment of inverse problems governed by linear operators, leading to model discovery from just a handful of noisy measurements.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

In AI for Physical Sciences

Citations in the corpus

fPINNs: Fractional Physics-Informed Neural Networks (2018, background): “For example, in [36], the parameters to be identified enter the negative log-likelihood function of the GP regression in the form of some extra hyperparameters of the covariance functions.”
Neural-net-induced Gaussian process regression for function approximation and PDE solution (2018, methodology): “Specifically, so far GP regression has been successfully applied to solution of PDEs including linear and nonlinear forward problems [6] as well as inverse problems [9,10].”
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations (2018, background): “One could avoid the ﬁrst drawback concerning numerical diﬀerentiation by assigning prior distributions in the forms of Gaussian processes [38, 39, 40, 41] or neural networks [35, 34] to the unknown solution u .”
Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems (2018, background): “…in the context of identifying nonlinear dynamics, there exist several deterministic and probabilistic tools including radial basis functions [6], neural networks [7], Gaussian processes [8, 9, 10, 11, 12], and nonlinear auto-regressive models such as NARMAX [13] and recurrent neural networks [14].”
Hidden physics models: Machine learning of nonlinear partial differential equations (2017, positive): “Similar to Raissi et al. [3, 4], we build upon the analytical property of Gaussian processes that the output of a linear system whose input is Gaussian distributed is again Gaussian.”
Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations (2017, background): “In this second part of our study, we shift our attention to the problem of data-driven discovery of partial differential equations [7, 8, 9].”
Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations (2017, background): “The first problem is that of predictive inference, filtering and smoothing, or data driven solutions of partial differential equations [9, 5] which states: given fixed model parameters λ what can be said about the unknown hidden state u(t, x) of the system? The second problem is that of learning, sy”
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations (2017, methodology): “Building upon the work of Raissi and coworkers [24, 25], we place a Gaussian process [26] prior on un−1, i.”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “The brilliance of the first PINN articles (Raissi et al, 2017c,d) lies in resurrecting the concept of optimizing a problem with a physical constraint by approximating the unknown function with a neural network (Dissanayake and Phan-Thien, 1994) and then extending this concept to a hybrid data-equati”
WEAK SINDY FOR PARTIAL DIFFERENTIAL EQUATIONS (2020, methodology): “Some of the primary techniques include Gaussian process regression [16], deep neural networks [18], Bayesian inference [28, 29] and classical methods from numerical analysis [8, 10, 25].”
Data Driven Governing Equations Approximation Using Deep Neural Networks (2018, methodology): “Methods have also been developed in conjunction with model selection approach ([28]), Koopman theory ([6]), and Gaussian process regression ([35]), to name a few.”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, background): “Physics-informed learning [7], introduced in a series of papers by Karniadakis’s group both for Gaussian-process regression [8,9] and physics-informed neural networks (PINNs) [10], can seamlessly integrate multifidelity/multi-”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “To this end, recent advances include using Gaussian process regression [17, 18, 19, 20, 21, 22] and deep neural networks (DNNs) [23, 24, 25, 12, 26] to solve forward problems as well as Bayesian estimation [27] and DNNs [28, 29] for inverse problems.”
B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data (2020, background): “However, vanilla GPR has diﬃculties in handling the nonlinearities when applied to solve PDEs, leading to restricted applications.”
Physical Symmetries Embedded in Neural Networks (2019, background): “(13)) by using NNs. Specifically, for each dimension k we have a system of two first order ODEs: q̇k = pk, ṗk = − ∂V (q) ∂qk .”
Multiscale modeling meets machine learning: What can we learn? (2019, background): “Recent examples of learning the underlying physics are the data-driven solution of problems in elasticity [23] and the data-driven discovery of partial differential equations for nonlinear dynamical systems [14, 95, 98].”
Machine Learning of Space-Fractional Differential Equations (2018, methodology): “In the aforementioned Gaussian process framework of [27], it is possible to treat time-derivatives by considering the data and equation in space-time.”