Deep learning for universal linear embeddings of nonlinear dynamics

By Bethany Lusch, J. Nathan Kutz, Steven L. Brunton. 2017

1,679 citations recorded by the source corpus.

Identifying coordinate transformations that make strongly nonlinear dynamics approximately linear has the potential to enable nonlinear prediction, estimation, and control using linear theory. The Koopman operator is a leading data-driven embedding, and its eigenfunctions provide intrinsic coordinates that globally linearize the dynamics. However, identifying and representing these eigenfunctions has proven challenging. This work leverages deep learning to discover representations of Koopman eigenfunctions from data. Our network is parsimonious and interpretable by construction, embedding the dynamics on a low-dimensional manifold. We identify nonlinear coordinates on which the dynamics are globally linear using a modified auto-encoder. We also generalize Koopman representations to include a ubiquitous class of systems with continuous spectra. Our framework parametrizes the continuous frequency using an auxiliary network, enabling a compact and efficient embedding, while connecting our models to decades of asymptotics. Thus, we benefit from the power of deep learning, while retaining the physical interpretability of Koopman embeddings. It is often advantageous to transform a strongly nonlinear system into a linear one in order to simplify its analysis for prediction and control. Here the authors combine dynamical systems with deep learning to identify these hard-to-find transformations.

Subfields: Computational Mechanics, Global and Planetary Change, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Global and Planetary Change

In Statistical and Nonlinear Physics

Citations in the corpus

Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow (2018, background): “In addition, learned Koopman operators [MWJK18] were proposed for representing temporal changes, whereas Lusch et al. are searching for representations of Koopman eigenfunctions to globally linearize dynamical systems using deep learning [LKB18].”
Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “spherical convolutional neural networks (CNNs) for the physical fields on unstructured grid [21], graph networks for object- and relationcentric representations of complex, dynamical systems [22], learning linear embeddings of nonlinear dynamics based on Koopman operator theory [23].”
Deep Generative Markov State Models (2018, background): “Finally, some of these models learn a linear model of the long-timescale dynamics [14, 19], but none of them provide a probabilistic dynamical model that can be employed in a Bayesian framework.”
Error bounds for kernel-based approximations of the Koopman operator (2023, background): “In light of this issue, representations of the Koopman operator on large approximation spaces have been considered in recent years, including deep neural networks [36, 39], tensor product spaces [26, 44], and reproducing kernel Hilbert spaces (RKHS) [63, 12, 25].”
Data-driven discovery of Koopman eigenfunctions for control (2017, methodology): “Nevertheless, the inverse mapping may be approximated using multidimensional scaling as in [57] or learned jointly with φ itself using autoencoders [73].”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “Though dynamic mode decomposition [197] is the most common technique for approximating the Koopman operator, many recent approaches have been made to approximate Koopman operator embeddings with deep learning models that outperform existing methods [170, 185, 191, 200, 209, 210, 260, 284, 307].”
Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks (2018, background): “Recently, data-driven approximations of Koopman operators, including dynamic mode decomposition [26], diﬀusion maps [26], delay co-ordinates [8, 4, 1], and neural networks [27, 23, 15, 14, 17], have gained signiﬁcant attention.”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “…done, the system can evolve in time using Recurrent Neural Networks (RNNs) such as LSTM (Wiewel et al., 2019) or even linear propagators based on the assumptions of learning Koopman operator (Morton et al., 2018; Li et al., 2020a; Lusch et al., 2018; Takeishi et al., 2017; Pan & Duraisamy, 2020).”
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics (2019, methodology): “More recently, in Lusch, Kutz, and Brunton (2018), a data-driven method based on the Koopman operator formalism (Koopman, 1931) was proposed, using feed-forward ANNs to identify an embedding space with linear dynamics that is then amenable to theoretical analysis.”
Variational Approach for Learning Markov Processes from Time Series Data (2017, methodology): “Some recent methods extend EDMDDL for modeling Koopman operators of deterministic systems [58,31,44], and solve the first problem by using the prediction error between the observed xt and that predicted by the low-dimensional model.”
Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition (2017, methodology): “After the first submission of this manuscript (in May 2017), several similar approaches to learning data transform for Koopman analysis have been proposed [41]–[45].”
A Deep Learning Approach for Objective-Driven All-Dielectric Metasurface Design (2019, positive): “It provides a promising solution to reducing time-consuming calculations and producing results with limited computational resources (14-17).”