Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining
By Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng and 1 more. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2024
444 citations recorded by the source corpus.
Accurate medical image segmentation demands the integration of multi-scale information, spanning from local features to global dependencies. However, it is challenging for existing methods to model long-range global information, where convolutional neural networks (CNNs) are constrained by their local receptive fields, and vision transformers (ViTs) suffer from high quadratic complexity of their attention mechanism. Recently, Mamba-based models have gained great attention for their impressive ability in long sequence modeling. Several studies have demonstrated that these models can outperform popular vision models in various tasks, offering higher accuracy, lower memory consumption, and less computational burden. However, existing Mamba-based models are mostly trained from scratch and do not explore the power of pretraining, which has been proven to be quite effective for data-efficient medical image analysis. This paper introduces a novel Mamba-based model, Swin-UMamba, designed specifically for medical image segmentation tasks, leveraging the advantages of ImageNet-based pretraining. Our experimental results reveal the vital role of ImageNet-based training in enhancing the performance of Mamba-based models. Swin-UMamba demonstrates superior performance with a large margin compared to CNNs, ViTs, and latest Mamba-based models. Notably, on AbdomenMRI, Encoscopy, and Microscopy datasets, Swin-UMamba outperforms its closest counterpart U-Mamba_Enc by an average score of 2.72%.
Subfields: Media Technology
View Paper: https://www.semanticscholar.org/paper/98a7444a221e27f51c89c58fa29a8a1e168c6d69
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Media Technology
Citations in the corpus
- LocalMamba: Visual State Space Model with Windowed Selective Scan (2024, methodology): “Leveraging Mamba’s foundation in generic models, new methodologies have been developed for visual tasks, such as image segmentation [30, 34, 40, 54] and image synthetic [14], showcasing the adaptability and effectiveness of visual Mamba models in addressing complex vision challenges.”
- PointMamba: A Simple State Space Model for Point Cloud Analysis (2024, background): “Besides, the great potential of Mamba motivates a series of work across diverse domains, including graph [50, 3], medical segmentation [36, 34], video understanding [30, 7] and generative models [29, 31].”