Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

By Kookjin Lee, Kevin T. Carlberg. Journal of Computational Physics, 2018

859 citations recorded by the source corpus.

Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) POD. Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov $n$-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov $n$-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov--Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Finally, we demonstrate the ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic $n$-width limitations of linear subspaces.

Subfields: Geophysics, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c7a60ef79b6a9c0c272dbf733541afa00b378f2e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Geophysics

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Enhancing computational fluid dynamics with machine learning (2021, background): “Lee and Carlberg [105] recently showed that deep convolutional autoencoders may be used to greatly improve the performance of classical ROM techniques based on linear subspaces [106, 107].”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, neutral): “4) [209] [16] [167] [138] [86] [210] [138] [241] [242] [278] [115] [210] [67]”
Data-driven discovery of coordinates and governing equations (2019, neutral): “The coordinate transformation is achieved using an autoen-coder network architecture.”
POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition (2021, methodology): “…reduced trial manifold where the approximation is sought (thus avoiding the linear superimposition of POD modes), either relying on a minimum residual formulation to derive the ROM [40, 41], or without considering an explicit parameter dependence in the diﬀerential problem that is considered [42].”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, neutral): “Reduced Order Models [244] [179] [101] [273] [225] [286] [173] [188] [10] [148] [125] [262] [171] [74] [188] [190] [125] [223] [224] [257] [101] [190] [57]”
A Comprehensive Deep Learning-Based Approach to Reduced Order Modeling of Nonlinear Time-Dependent Parametrized PDEs (2020, methodology): “For instance, a projection-based ROM technique has been introduced in [37], in which the FOM system is projected onto a nonlinear trial manifold identified by means of the decoder function of a convolutional autoencoder.”
Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data (2020, methodology): “To overcome the linearity limitations of POD and DMD, deep learning based approaches (e.g., autoencoder neural networks) have been recently developed to extract nonlinear latent representations of the ﬂow ﬁeld from massive oﬄine data [13].”
Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders (2020, methodology): “[42] also uses a CAE to nonlinearly embed states, but solves the governing equations on the nonlinear manifold defined”