Efficient training of physics‐informed neural networks via importance sampling

By Mohammad Amin Nabian, Rini Jasmine Gladstone, Hadi Meidani. Comput. Aided Civ. Infrastructure Eng., 2021

366 citations recorded by the source corpus.

Physics‐informed neural networks (PINNs) are a class of deep neural networks that are trained, using automatic differentiation, to compute the response of systems governed by partial differential equations (PDEs). The training of PINNs is simulation free, and does not require any training data set to be obtained from numerical PDE solvers. Instead, it only requires the physical problem description, including the governing laws of physics, domain geometry, initial/boundary conditions, and the material properties. This training usually involves solving a nonconvex optimization problem using variants of the stochastic gradient descent method, with the gradient of the loss function approximated on a batch of collocation points, selected randomly in each iteration according to a uniform distribution. Despite the success of PINNs in accurately solving a wide variety of PDEs, the method still requires improvements in terms of computational efficiency. To this end, in this paper, we study the performance of an importance sampling approach for efficient training of PINNs. Using numerical examples together with theoretical evidences, we show that in each training iteration, sampling the collocation points according to a distribution proportional to the loss function will improve the convergence behavior of the PINNs training. Additionally, we show that providing a piecewise constant approximation to the loss function for faster importance sampling can further improve the training efficiency. This importance sampling approach is straightforward and easy to implement in the existing PINN codes, and also does not introduce any new hyperparameter to calibrate. The numerical examples include elasticity, diffusion, and plane stress problems, through which we numerically verify the accuracy and efficiency of the importance sampling approach compared to the predominant uniform sampling approach.

Subfields: Artificial Intelligence, Mechanical Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/af68d37e2721abe702bacfcea5111f9ec59012ce

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Mechanical Engineering

Gianluigi Rozza

In Statistical and Nonlinear Physics

Citations in the corpus

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems (2021, neutral): “…distributed in the domain or grid points on a lattice, and other methods of training point sampling and distribution have been proposed to achieve better accuracy when using the same number of training points, such as the residual-based adaptive reﬁnement (RAR) [3] and importance sampling [12].”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, background): “There are some PINN approaches that propose to select collocations points in specific areas of the space-time domain (Nabian et al, 2021); this should be investigated as well.”
CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method (2021, background): “In general, a huge amount of collocation points is required for matching the differential equations in order to train a good PINN model [41,42].”