KAN: Kolmogorov-Arnold Networks

By Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark. International Conference on Learning Representations, 2024

1,915 citations recorded by the source corpus.

Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

View Paper: https://www.semanticscholar.org/paper/14fdab35cc6288083a38a92392af3f1da0b95a90

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wuyang Li
2. Yixuan Yuan
3. Xinyu Liu
4. Chenxin Li
5. Hengyu Liu
6. Cheng Wang

Citations in the corpus

• U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation (2024, methodology): “As Kolmogorov–Arnold Network (KAN) [55] has been emgerged as a promosing alternative for MLP and demonstrates its precision, efficiency, and interpretability, we believe now is the right time to open up the exploration of its broader applications in vision backbones.”