GNOT: A General Neural Operator Transformer for Operator Learning
By Zhongkai Hao, Chengyang Ying, Zhengyi Wang, Hang Su, Yinpeng Dong, Songming Liu, Ze Cheng, Jun Zhu, Jian Song. International Conference on Machine Learning, 2023
432 citations recorded by the source corpus.
Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods. Our code and data are publicly available at \url{https://github.com/thu-ml/GNOT}.
Subfields: Statistical and Nonlinear Physics
View Paper: https://www.semanticscholar.org/paper/6e036e28e7af03bfcdd98ffa254df6644f7657c5
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistical and Nonlinear Physics
- Anima Anandkumar
- Zong-Yi Li
- Nikola B. Kovachki
- Kamyar Azizzadenesheli
- Jean Kossaifi
- Haixu Wu
- Jianmin Wang
- Huakun Luo
Citations in the corpus
- Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, methodology): “This includes the Operator Transformer (OFormer) [39], Mesh-Independent Neural Operator (MINO) [40], and the General Neural Operator Transformer (GNOT) [41].”
- Transolver: A Fast Transformer Solver for PDEs on General Geometries (2024, methodology): “To clearly benchmark our model among multifarious neural operators, we first experiment on six well-established datasets, which can conveniently build a complete leaderboard from previous papers (Li et al., 2022; Wu et al., 2023; Hao et al., 2023).”