Inferring solutions of differential equations using noisy multi-fidelity data

By Maizar Raissi, Paris Perdikaris, G. Em Karniadakis. 2016

319 citations recorded by the source corpus.

For more than two centuries, solutions of differential equations have been obtained either analytically or numerically based on typically well-behaved forcing and boundary conditions for well-posed problems. We are changing this paradigm in a fundamental way by establishing an interface between probabilistic machine learning and differential equations. We develop data-driven algorithms for general linear equations using Gaussian process priors tailored to the corresponding integro-differential operators. The only observables are scarce noisy multi-fidelity data for the forcing and solution that are not required to reside on the domain boundary. The resulting predictive posterior distributions quantify uncertainty and naturally lead to adaptive solution refinement via active learning. This general framework circumvents the tyranny of numerical discretization as well as the consistency and stability issues of time-integration, and is scalable to high-dimensions.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Neural-net-induced Gaussian process regression for function approximation and PDE solution (2018, methodology): “Denoting by Xu (Nu by din matrix) the collection of the boundary evaluation points {xu, j} and by X f (N f by din matrix) the collection of domain evaluation points {x f , j}, we formulate the covariance matrix in (18) as [6] K oo = [ ku(Xu, Xu) + σ 2 noise,u I Lx′ku(Xu, X f ) Lxku(X f , Xu) LxLx′ku”
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations (2018, background): “One could avoid the ﬁrst drawback concerning numerical diﬀerentiation by assigning prior distributions in the forms of Gaussian processes [38, 39, 40, 41] or neural networks [35, 34] to the unknown solution u .”
Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems (2018, background): “…in the context of identifying nonlinear dynamics, there exist several deterministic and probabilistic tools including radial basis functions [6], neural networks [7], Gaussian processes [8, 9, 10, 11, 12], and nonlinear auto-regressive models such as NARMAX [13] and recurrent neural networks [14].”
Hidden physics models: Machine learning of nonlinear partial differential equations (2017, positive): “Similar to Raissi et al. [3, 4], we build upon the analytical property of Gaussian processes that the output of a linear system whose input is Gaussian distributed is again Gaussian.”
Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations (2017, background): “Through the lens of different benchmark problems, we highlighted the key features of physics informed neural networks in the context of data-driven solutions of partial differential equations [5, 6].”
Adversarial Uncertainty Quantification in Physics-Informed Neural Networks (2018, methodology): “In contrast to previous approaches to inferring solutions of partial differential equations from data [44, 45, 46, 47], the proposed methodology does not rely on Gaussian assumptions, and it can directly tackle nonlinear problems without any need for linearization.”
Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations (2017, background): “The first problem is that of predictive inference, filtering and smoothing, or data driven solutions of partial differential equations [9, 5] which states: given fixed model parameters λ what can be said about the unknown hidden state u(t, x) of the system? The second problem is that of learning, sy”
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations (2017, methodology): “Building upon the work of Raissi and coworkers [24, 25], we place a Gaussian process [26] prior on un−1, i.”
Machine learning of linear differential equations using Gaussian processes (2017, background): “[38] on inferring solutions of linear differential equations from noisy multi-fidelity data.”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, methodology): “The brilliance of the first PINN articles (Raissi et al, 2017c,d) lies in resurrecting the concept of optimizing a problem with a physical constraint by approximating the unknown function with a neural network (Dissanayake and Phan-Thien, 1994) and then extending this concept to a hybrid data-equati”
Bayesian Probabilistic Numerical Methods (2017, background): “(2017) Probabilistic Meshless Methods (Owhadi, 2015, 2017; Cockayne et al., 2016; Raissi et al., 2016) Dx + discretisation error Conrad et al.”
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations (2019, background): “The first problem is that of inference, filtering and smoothing, or data-driven solutions of partial differential equations [4,8] which states: given fixed model parameters λ what can be said about the unknown hidden state u(t, x) of the system? The second problem is that of learning, system identif”
A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems (2019, methodology): “Although great progress has already been made, the multi-ﬁdelity approaches based on GPR still have some limitations, e.g., approximations of discontinuous functions [7], high-dimensional problems [9], and inverse problems with strong nonlinearities (i.e., nonlinear partial diﬀerential equations) [8”
Adaptive activation functions accelerate convergence in deep and physics-informed neural networks (2019, methodology): “Further, their method was extended to nonlinear problems by Raissi and Karniadakis [23] and Raissi et al [24] in the context of solution inference and system identification.”
A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime (2019, unknown): “(2018) which are trained by minimization of a loss function augmented by the residuals of the governing equations (Raissi et al., 2017) and (Raissi et al.”
Hamiltonian Neural Networks for solving differential equations (2020, background): “models [6]–[9], discover differential equations (DEs) [10]– [13], and find approximate solutions for those equations [14], [15].”