Deep learning for physical processes: incorporating prior scientific knowledge

By Emmanuel de Bézenac, Arthur Pajot, Patrick Gallinari. 2017

364 citations recorded by the source corpus.

We consider the use of deep learning methods for modeling complex phenomena like those occurring in natural physical processes. With the large amount of data gathered on these phenomena the data intensive paradigm could begin to challenge more traditional approaches elaborated over the years in fields like maths or physics. However, despite considerable successes in a variety of application domains, the machine learning field is not yet ready to handle the level of complexity required by such problems. Using an example application, namely sea surface temperature prediction, we show how general background knowledge gained from the physics could be used as a guideline for designing efficient deep learning models. In order to motivate the approach and to assess its generality we demonstrate a formal link between the solution of a class of differential equations underlying a large family of physical phenomena and the proposed model. Experiments and comparison with series of baselines including a state of the art numerical approach is then provided.

Subfields: Artificial Intelligence, Atmospheric Science, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Atmospheric Science

In Statistical and Nonlinear Physics

Citations in the corpus

Differentiable Physics-informed Graph Networks (2019, background): “Although these works leveraged physical knowledge, they are limited because they require all physics behind given data to be explicitly known. de Bezenac et al. (2018) considered a similar problem as ours.”
Enforcing Analytic Constraints in Neural Networks Emulating Physical Systems. (2019, background): “Despite the need for hybrid-NNs to model complex physical systems [1–3], enforcing hard constraints [4] has been limited to physical systems governed by specific equations, such as advection [5, 6] or Reynolds-averaged Navier-Stokes [7, 8] equations.”
Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics (2020, background): “In common, the methods in Chen et al. (2015); de Bezenac et al. (2018); Lutter et al. (2019) are not efﬁciently applicable to sparsely discretized input as only a small number of data points are available and continuous properties on given space are not easily recovered.”
Data-driven synthesis of smoke flows with CNN-based feature descriptors (2017, positive): “While previous work shows that warping layers are useful in motion field learning [Chen et al. 2017; de Bezenac et al. 2017], our work targets the opposite direction: by providing our networks with velocities, warping layers can likewise improve the training of temporally coherent content…”
Can deep learning beat numerical weather prediction? (2021, background): “De Bézenac et al. [127] embedded an advection diffusion equation in their loss function to predict sea surface temperatures and thereby improved the predictive capabilities of their model.”
Physics-informed machine learning: case studies for weather and climate modelling (2021, background): “De Bezenac et al. [59] use a warping scheme based on the advection-diffusion equation that preserves spatial coherence in conjunction with a CNN to predict the evolution of sea surface temperatures.”
Learning Mesh-Based Simulation with Graph Networks (2020, background): “Learned simulations can be useful for real-world predictions where the physical model, parameters or boundary conditions are not fully known [11].”
Disentangling Physical Dynamics From Unknown Factors for Unsupervised Video Prediction (2020, methodology): “Recently, interesting connections between residual networks and PDEs have been drawn [71, 37, 8], enabling to design physicallyconstrained machine learning frameworks [48, 11, 55, 52].”
Towards Physics-informed Deep Learning for Turbulent Flow Prediction (2019, methodology): “For example, [8] proposed a warping scheme to predict the sea surface temperature, but only considered the linear advection-diffusion equation.”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, neutral): “Solve PDEs [147] [65] [140] [9] [215] [42] [264] [106] [235] [107] [131] [209] [234] [279] [281] [60] [177] [208] [231] [301] [71] [89] [129] [268] [194] [176] [16] [48] [218] [44] [60] [172] [235] [19] [51] [131] [76] [278] [194] [176] [206] [166]”
Learning data-driven discretizations for partial differential equations (2018, background): “Recent works have applied machine learning to partial differential equations (PDEs), either focusing on speed (8–10) or recovering unknown dynamics (11, 12).”
Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems (2019, methodology): “tions [44] are used in oceanography to model the evolution of sea surface temperatures.”
PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network (2018, background): “In [14], the authors studied the problem of sea surface temperature prediction (SSTP).”