Multiwavelet-based Operator Learning for Differential Equations

By Gaurav Gupta, Xiongye Xiao, Paul C. Bogdan. Neural Information Processing Systems, 2021

324 citations recorded by the source corpus.

The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a \textit{multiwavelet-based neural operator learning scheme} that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a ($2X-10X$) improvement ($0.0018$ ($0.0033$) relative $L2$ error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/90457535d179af22c182bba257b07651903f014e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Transolver: A Fast Transformer Solver for PDEs on General Geometries (2024, methodology): “Besides, WMT (Gupta et al., 2021) introduces multiscale wavelet bases to capture the complex correlations at various scales.”
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting (2022, methodology): “Inspired by (Gupta et al., 2021), we adapt a non-standard wavelet representation to reduce the model complexity.”
FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting (2022, background): “OPs (including Legendre, Laguerre, Chebyshev, etc.) and other orthogonal basis (Fourier and Multiwavelets) are widely studied in numerous ﬁelds and recently brought in deep learning Wang et al. (2018); Gupta et al. (2021); Zhou et al. (2022); Voelker et al. (2019); Gu et al. (2020).”
GNOT: A General Neural Operator Transformer for Operator Learning (2023, background): “Some works also propose variants of FNO from other aspects (Gupta et al., 2021; Wen et al., 2022; Tran et al., 2021).”
Transformer for Partial Differential Equations' Operator Learning (2022, methodology): “In addition, this usually requires the model to have larger number of parameters in order to model the extra degree of freedom (Li et al., 2021a; Gupta et al., 2021).”
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers (2021, background): “There are still important avenues to explore for the future work such as exploring alterantives for the DFT such as the Wavelet transform to better capture locality as in Gupta et al. (2021).”
Choose a Transformer: Fourier or Galerkin (2021, background): “Data-driven resolution-invariant operator learning is a booming new research direction [60, 5, 56, 64, 90, 57, 61, 91, 37, 74].”