Learning data-driven discretizations for partial differential equations

By Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, Michael P. Brenner. Proceedings of the National Academy of Sciences of the United States of America, 2018

609 citations recorded by the source corpus.

Significance In many physical systems, the governing equations are known with high confidence, but direct numerical solution is prohibitively expensive. Often this situation is alleviated by writing effective equations to approximate dynamics below the grid scale. This process is often impossible to perform analytically and is often ad hoc. Here we propose data-driven discretization, a method that uses machine learning to systematically derive discretizations for continuous physical systems. On a series of model problems, data-driven discretization gives accurate solutions with a dramatic drop in required resolution. The numerical solution of partial differential equations (PDEs) is challenging because of the need to resolve spatiotemporal features over wide length- and timescales. Often, it is computationally intractable to resolve the finest features in the solution. The only recourse is to use approximate coarse-grained representations, which aim to accurately represent long-wavelength dynamics while properly accounting for unresolved small-scale physics. Deriving such coarse-grained equations is notoriously difficult and often ad hoc. Here we introduce data-driven discretization, a method for learning optimized approximations to PDEs based on actual solutions to the known underlying equations. Our approach uses neural networks to estimate spatial derivatives, which are optimized end to end to best satisfy the equations on a low-resolution grid. The resulting numerical methods are remarkably accurate, allowing us to integrate in time a collection of nonlinear equations in 1 spatial dimension at resolutions 4× to 8× coarser than is possible with standard finite-difference methods.

Subfields: Atmospheric Science, Computational Mechanics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/9a43842478bf9d1aff71e964f584af4eeb196073

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, neutral): “Solve PDEs [147] [65] [140] [9] [215] [42] [264] [106] [235] [107] [131] [209] [234] [279] [281] [60] [177] [208] [231] [301] [71] [89] [129] [268] [194] [176] [16] [48] [218] [44] [60] [172] [235] [19] [51] [131] [76] [278] [194] [176] [206] [166]”
Learning to Control PDEs with Differentiable Physics (2020, methodology): “In this context, several works have proposed methods for improving the solution of PDE problems (Long et al., 2018; Bar-Sinai et al., 2019; Hsieh et al., 2019) or used PDE formulations for unsupervised optimization (Raissi et al.”
Physics-informed machine learning: case studies for weather and climate modelling (2021, methodology): “[64] use ML to learn discretizations based on actual solutions to the known underlying equations that are significantly more accurate and faster than existing methods.”
Hamiltonian Neural Networks for solving differential equations (2020, background): “Recently, machine learning models and particularly neural networks (NNs) have been used to explore those data and forecast the future behavior of complex dynamical systems [1–4], improve turbulence models [5–7, 25], discover differential equations (DEs) [8–11] and find approximate solutions for thos”
Data-driven discovery of coordinates and governing equations (2019, methodology): “Numerous recent approaches leverage machine learning to identify dynamical systems models from data [19–26], with many using neural networks to model time-series data [18, 27–38].”
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting (2020, background): “However, research into physical machine learning is ongoing and might offer new opportunities in the near future (Bar-Sinai et al., 2019; Beucler et al., 2019).”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, neutral): “5) [226] [251] [301] [303] [70] [196] [228] [248] [318] [82] [102] [144] [292] [216] [195] [23] [90] [320] [154] [58] [236] [53] [70] [192] [252] [26] [61] [149] [88] [299] [216] [195] [225] [145] [198] [171] [44] [188]”
Factorized Fourier Neural Operators (2021, methodology): “Hybrid methods typically aim to speed up traditional numerical solvers by using lower resolution grids (Bar-Sinai et al., 2019; Um et al., 2020; Kochkov et al., 2021), or by replacing computationally expensive parts of the solver with learned alternatives Tompson et al. (2017); Obiols-Sales et al. (”
Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers (2020, methodology): “We mimic the setup from previous work [4] to inject energy into the system via a forcing term with a spectrum of sine waves.”
Machine learning–accelerated computational fluid dynamics (2021, positive): “We use data-driven discretizations (36, 37) to interpolate differential operators onto a coarse mesh with high accuracy (Fig.”
Enhancing computational fluid dynamics with machine learning (2021, methodology): “Bar-Sinai et al. [21] proposed a technique based on deep learning to estimate spatial derivatives in low-resolution grids, outperforming standard ﬁnite-difference methods.”
Message Passing Neural PDE Solvers (2022, methodology): “As mentioned in Bar-Sinai et al. (2019), both FDM and FVM are linear methods, which estimate n th -order point-wise function derivatives as for appropriately chosen coefﬁcients α ( n ) j , where N ( i ) is the neighborhood of cell i .”
Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics (2020, methodology): “[5] Y.”