Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation

By Junde Wu, Rao Fu, Huihui Fang, Yuanpei Liu, Zhao-Yang Wang, Yanwu Xu, Yueming Jin, Tal Arbel. Medical Image Analysis, 2023

975 citations recorded by the source corpus.

The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation due to its impressive capabilities in various segmentation tasks and its prompt-based interface. However, recent studies and individual experiments have shown that SAM underperforms in medical image segmentation due to the lack of medical-specific knowledge. This raises the question of how to enhance SAM's segmentation capability for medical images. We propose the Medical SAM Adapter (Med-SA), which is one of the first methods to integrate SAM into medical image segmentation. Med-SA uses a light yet effective adaptation technique instead of fine-tuning the SAM model, incorporating domain-specific medical knowledge into the segmentation model. We also propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation. Comprehensive evaluation experiments on 17 medical image segmentation tasks across various modalities demonstrate the superior performance of Med-SA while updating only 2% of the SAM parameters (13M). Our code is released at https://github.com/KidsWithTokens/Medical-SAM-Adapter.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/49f9882d5fd442f02f9c9dff780336f6dce2da4f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Foundation Models Defining a New Era in Vision: A Survey and Outlook (2025, neutral): “Further, such a generic visual prompt-based segmentation model can be adapted for speciﬁc downstream tasks such as medical image segmentation [12], [13], video object segmentation [14], robotics [15], and remote sensing [16].”
SAM 2: Segment Anything in Images and Videos (2024, background): “…of SAM led to its adoption in a wide range of applications, such as medical imaging (Ma et al., 2024; Deng et al., 2023; Mazurowski et al., 2023; Wu et al., 2023a), remote sensing (Chen et al., 2024; Ren et al., 2024), motion segmentation (Xie et al., 2024), and camouflaged object detection…”
Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions (2024, methodology): “Rather than fully adjusting all parameters, Wu et al. [14] propose Medical SAM Adapter (Med-SA), which maintains the pre-trained SAM parameters frozen while integrating low-rank adaptation (LoRA) modules [67] to the designated positions.”
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis (2023, background): “They include X-rays, Computed Tomography (CT), Magnetic Resonance Image (MRI), Ultrasound imaging, and Positron Emission Tomography (PET).”