Parallel Physics-Informed Neural Networks via Domain Decomposition

By Khemraj Shukla, Ameya Dilip Jagtap, G. Em Karniadakis. Journal of Computational Physics, 2021

416 citations recorded by the source corpus.

We develop a distributed framework for the physics-informed neural networks (PINNs) based on two recent extensions, namely conservative PINNs (cPINNs) and extended PINNs (XPINNs), which employ domain decomposition in space and in time-space, respectively. This domain decomposition endows cPINNs and XPINNs with several advantages over the vanilla PINNs, such as parallelization capacity, large representation capacity, efficient hyperparameter tuning, and is particularly effective for multi-scale and multi-physics problems. Here, we present a parallel algorithm for cPINNs and XPINNs constructed with a hybrid programming model described by MPI $+$ X, where X $\in \{\text{CPUs},~\text{GPUs}\}$. The main advantage of cPINN and XPINN over the more classical data and model parallel approaches is the flexibility of optimizing all hyperparameters of each neural network separately in each subdomain. We compare the performance of distributed cPINNs and XPINNs for various forward problems, using both weak and strong scalings. Our results indicate that for space domain decomposition, cPINNs are more efficient in terms of communication cost but XPINNs provide greater flexibility as they can also handle time-domain decomposition for any differential equations, and can deal with any arbitrarily shaped complex subdomains. To this end, we also present an application of the parallel XPINN method for solving an inverse diffusion problem with variable conductivity on the United States map, using ten regions as subdomains.

Subfields: Artificial Intelligence, Mechanical Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/ab82d2e0bd23f55b405e3c0a593417fa5fcdb7f6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Mechanical Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Analyses of internal structures and defects in materials using physics-informed neural networks (2022, background): “9 of 12 (49) have been proposed to accelerate the learning process of PINNs by using multiple CPUs and graphics processing units (GPUs) and introducing parallel algorithms.”
Physics-informed neural networks for inverse problems in supersonic flows (2022, methodology): “In recent years physics-informed neural networks (PINNs) [1] emerged as an alternative simple method to solve many problems in computational science and engineering, see, for example [2, 3, 4, 5, 6, 7, 8, 9, 10, 4, 11, 12, 13, 14, 15].”
Physics-informed neural networks (PINNs) for fluid mechanics: a review (2021, methodology): “A general parallel implementation of PINNs with domain decomposition for flow problems is presented in [27]; the NVIDIA library SimNet [28] is also a very efficient implementation of PINNs.”
Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations (2021, background): “In follow up work Shukla et al. [2021] showed their approach can be parallelised to multiple GPUs, decreasing training times.”
Physics‐Informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions (2021, methodology): “The implementation of PINNs with a domain decomposition in space and space-time domain is successfully studied by Jagtap et al. (2020) and Jagtap and Karniadakis (2020), which is further extended to a multi-GPU platform by Shukla et al. (2021).”