Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator.

By Qianxao Li, Felix Dietrich, Erik M. Bollt, Ioannis G. Kevrekidis. 2017

512 citations recorded by the source corpus.

Numerical approximation methods for the Koopman operator have advanced considerably in the last few years. In particular, data-driven approaches such as dynamic mode decomposition (DMD)51 and its generalization, the extended-DMD (EDMD), are becoming increasingly popular in practical applications. The EDMD improves upon the classical DMD by the inclusion of a flexible choice of dictionary of observables which spans a finite dimensional subspace on which the Koopman operator can be approximated. This enhances the accuracy of the solution reconstruction and broadens the applicability of the Koopman formalism. Although the convergence of the EDMD has been established, applying the method in practice requires a careful choice of the observables to improve convergence with just a finite number of terms. This is especially difficult for high dimensional and highly nonlinear systems. In this paper, we employ ideas from machine learning to improve upon the EDMD method. We develop an iterative approximation algorithm which couples the EDMD with a trainable dictionary represented by an artificial neural network. Using the Duffing oscillator and the Kuramoto Sivashinsky partical differential equation as examples, we show that our algorithm can effectively and efficiently adapt the trainable dictionary to the problem at hand to achieve good reconstruction accuracy without the need to choose a fixed dictionary a priori. Furthermore, to obtain a given accuracy, we require fewer dictionary terms than EDMD with fixed dictionaries. This alleviates an important shortcoming of the EDMD algorithm and enhances the applicability of the Koopman framework to practical problems.

Subfields: Aerospace Engineering, Computational Mechanics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Statistical and Nonlinear Physics

Citations in the corpus

Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems (2017, methodology): “showed that a shallow 3-layer neural network with Tikhonov regularization using the tanh activation function, can also learn oscillatory dynamics for the Duffing oscillator [10].”
Universal Differential Equations for Scientific Machine Learning (2020, methodology): “Many methods produce non-symbolic representations by learning functional representations [67, 68] or through dynamic mode decomposition (DMD, eDMD) [69, 70, 71, 72].”
Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems (2019, methodology): “Koopman operator theory, which states that every nonlinear 39 dynamical system can be exactly described by an infinite-dimensional linear 40 operator that acts on scalar observables of the system [18], has been used to 41 extend DMD to fit linear models for nonlinear systems in observable space 42 d”
Koopman Lyapunov‐based model predictive control of nonlinear chemical process systems (2019, unknown): “Several other approaches propose using deep learning methods to discover these dictionaries.(33,34) Nevertheless, when the system model is known or a limited knowledge on the functional forms present in the model is available, it can be readily incorporated in the dictionary as will be shown with an”
Machine learning for molecular simulation (2019, methodology): “EDMD with dictionary learning [117] uses a similar architecture as VAMPnets, but is optimized by minimizing the regression error in latent space.”
Data-driven approximation of the Koopman generator: Model reduction, system identification, and control (2019, background): “Algorithms such as DMD [1, 2], EDMD [3, 4], SINDy [5], and their various kernel-[3, 6, 7], tensor-[8, 9, 10], or neural network-based [11, 12, 13] extensions and generalizations have been successfully applied to a plethora of diﬀerent problems, including molecular and ﬂuid dynamics, meteorology,…”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, methodology): “Though dynamic mode decomposition [197] is the most common technique for approximating the Koopman operator, many recent approaches have been made to approximate Koopman operator embeddings with deep learning models that outperform existing methods [170, 185, 191, 200, 209, 210, 260, 284, 307].”
Variational Approach for Learning Markov Processes from Time Series Data (2017, methodology): “The major advantages of the nonlinear TCCA over EDMD-DL are: First, the uninformative model with zero regression error can be systematically excluded without any extra constraints on features.”
Data-driven discovery of coordinates and governing equations (2019, background): “Several recent works have focused on the use of deep learning methods to discover the proper coordinates for DMD and extended DMD [31, 32, 79].”
Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition (2017, methodology): “After the first submission of this manuscript (in May 2017), several similar approaches to learning data transform for Koopman analysis have been proposed [41, 42, 43, 44, 45].”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “Though dynamic mode decomposition [178] is the most common technique for approximating the Koopman operator, many recent approaches have been made to approximate Koopman operator embeddings with deep learning models that outperform existing methods [150, 163, 171, 180, 188, 190, 244, 262, 286].”
Linearly-Recurrent Autoencoder Networks for Learning Dynamics (2017, unknown): “le. Training proceeds by iterating two steps: (1) Fix and optimize K by explicit solution of the least squares problem; Then (2) x K and optimize by gradient descent. The algorithm implemented in [14] is summarized in Algorithm 1. 10 S. E. OTTO AND C. W. ROWLEY Algorithm 1 EDMD with dictionary lea”