A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems

By Xuhui Meng, G. Em Karniadakis. 2019

673 citations recorded by the source corpus.

Abstract Currently the training of neural networks relies on data of comparable accuracy but in real applications only a very small set of high-fidelity data is available while inexpensive lower fidelity data may be plentiful. We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function that is not easy to approximate with other methods, e.g. Gaussian process regression. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, by decomposing the correlation into a linear and nonlinear part, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can: (1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data. Despite the relatively low dimension and limited number of fidelities (two-fidelity levels) for the benchmark problems in the present study, the proposed model can be readily extended to very high-dimensional regression and classification problems involving multi-fidelity data.

Subfields: Aerospace Engineering, Statistical and Nonlinear Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs (2019, background): “Therefore, PPINN is able to tackle multi-fidelity modeling of inverse physical problems [25].”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, background): “…some other researches on PINNs focused on the development of the neural network architecture and the training, e.g., using multi-ﬁdelity framework [34], adaptive activation functions [35] and dynamic weights of the loss function [36], hard constraints [37] and CNN-based network architectures…”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, neutral): “Solve PDEs [147] [65] [140] [9] [215] [42] [264] [106] [235] [107] [131] [209] [234] [279] [281] [60] [177] [208] [231] [301] [71] [89] [129] [268] [194] [176] [16] [48] [218] [44] [60] [172] [235] [19] [51] [131] [76] [278] [194] [176] [206] [166]”
Review of multi-fidelity models (2016, methodology): “By utilizing pre-trained neural networks that have learned representations from HFMs on related tasks, this approach allows efficient knowledge sharing between fidelity levels, achieving better performance while saving computational effort [149, 36, 84].”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, methodology): “We also have to facilitate multi-fidelity data that not only include multi-resolution data but also data corresponding to different levels of physical complexity, and hence training of NN requires special multi-fidelity methods such as in [34, 35, 36].”
PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain (2020, methodology): “This idea has also been extended to utilize multi-fidelity datasets [34] and solve system identification problems [35–37].”
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data (2019, background): “The PINN approach has been recently extended to assimilate multi-fidelity training data [38], and its UQ analyses have been explored based on arbitrary polynomial chaos [39] and adversarial inference [40].”