Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks

By Pantelis R. Vlachas, Wonmin Byeon, Zhong Y. Wan, Themistoklis P. Sapsis, Petros Koumoutsakos. Proceedings of the Royal Society A, 2018

561 citations recorded by the source corpus.

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto–Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM–LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/f772c778135e095d7f77d6536b673958caa7c2df

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Data-driven discovery of coordinates and governing equations (2019, unknown): “LSTMs have recently been used for forecasting of chaotic dynamical systems (20).”
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics (2019, methodology): “Furthermore, following Vlachas et al. (2018) we add Gaussian noise sampled from N (0 , κ n σ ) to the training data, where σ is the standard deviation of the data.”
Digital Twin: Values, Challenges and Enablers From a Modeling Perspective (2019, background): “More work on hybrid analytics or hybrid physics-AI modeling can be found in [340]–[344].”
SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics (2020, background): “, system identification) is a rapidly developing field [1], with a range of techniques including classic linear approaches [2, 3], dynamic mode decomposition (DMD) [4, 5] and Koopman theory [6–9], nonlinear autoregressive models [10, 11], neural networks [12–20], Gaussian process regression [21, 22]”
Neural operators for accelerating scientific simulations and design (2023, unknown): “Previous developments such as sparse representation [7], recurrent neural networks [8], reservoir computing [9] and others have shown promise in modeling dynamical systems.”
Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders (2020, background): “s and the latter are used for the temporal evolution of these compressed representations. We note that, recently, LSTMs have become popular for the non-intrusive characterization of dynamical systems [34, 35, 36, 37, 38, 39] although most such studies perform latent space embedding through the use o”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, unknown): “1) [266] [10] [219] [158] [140] [182] [203] [76] [86] [130] [174] [204] [231] [313] [121] [125] [187] [92] [306] [180] [298] [184] [267] [39] [247] [133] [130] [231] [298] [173] [66] [24] [255] [12] [17] [203] [204] [213] [239] [125] [281] [214] [176] [249] [310] [309] [311] [245] [78] [266] [297] [”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “Estimator quality at a given time and location can also be used to determine whether a prediction comes from the physical model or the ML model, which was shown in Chen et al. [50] for air pollution estimation and in Vlachas et al. [255] for dynamical system forecasting more generally.”