Linearly-Recurrent Autoencoder Networks for Learning Dynamics

By Samuel E. Otto, Clarence W. Rowley. 2017

409 citations recorded by the source corpus.

This paper describes a method for learning low-dimensional approximations of nonlinear dynamical systems, based on neural-network approximations of the underlying Koopman operator. Extended Dynamic Mode Decomposition (EDMD) provides a useful data-driven approximation of the Koopman operator for analyzing dynamical systems. This paper addresses a fundamental problem associated with EDMD: a trade-off between representational capacity of the dictionary and over-fitting due to insufficient data. A new neural network architecture combining an autoencoder with linear recurrent dynamics in the encoded state is used to learn a low-dimensional and highly informative Koopman-invariant subspace of observables. A method is also presented for balanced model reduction of over-specified EDMD systems in feature space. Nonlinear reconstruction using partially linear multi-kernel regression aims to improve reconstruction accuracy from the low-dimensional state when the data has complex but intrinsically low-dimensional structure. The techniques demonstrate the ability to identify Koopman eigenfunctions of the unforced Duffing equation, create accurate low-dimensional models of an unstable cylinder wake flow, and make short-time predictions of the chaotic Kuramoto-Sivashinsky equation.

Subfields: Computational Mechanics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Generative Markov State Models (2018, background): “Finally, some of these models learn a linear model of the long-timescale dynamics [14, 19], but none of them provide a probabilistic dynamical model that can be employed in a Bayesian framework.”
Enhancing computational fluid dynamics with machine learning (2021, background): “DMD has also been extended to nonlinear coordinate embed-dings through the use of deep-autoencoder networks [126–130].”
Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems (2017, methodology): “The ideas in this paper are complementary to the recent work presented in [10] and [11], [12], [13].”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “Though dynamic mode decomposition [197] is the most common technique for approximating the Koopman operator, many recent approaches have been made to approximate Koopman operator embeddings with deep learning models that outperform existing methods [170, 185, 191, 200, 209, 210, 260, 284, 307].”
Variational Approach for Learning Markov Processes from Time Series Data (2017, methodology): “Some recent methods extend EDMDDL for modeling Koopman operators of deterministic systems [58,31,44], and solve the first problem by using the prediction error between the observed xt and that predicted by the low-dimensional model.”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “This encoder-decoder architecture is further expanded on by Otto et al. [188], where they go beyond linear reconstruction using Koopman modes in the decoder step to an architecture that does nonlinear reconstruction in order to learn ever lower dimensional Koopman subspaces.”
Data-driven discovery of coordinates and governing equations (2019, methodology): “In addition to methods for discovering linear embeddings [18, 33], deep learning has also been used for parameter estimation of PDEs [36, 37].”
Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition (2017, methodology): “After the first submission of this manuscript (in May 2017), several similar approaches to learning data transform for Koopman analysis have been proposed [41, 42, 43, 44, 45].”
Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders (2018, background): “[69, 56, 47, 53], which apply autoencoders to learn approximate invariant subspaces of the Koopman operator; and Ref.”
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies (2022, methodology): “The most popular DL model for this is the autoencoder (Masti and Bemporad, 2021; Li and Yang, 2021; Otto and Rowley, 2019).”