Faster Segment Anything: Towards Lightweight SAM for Mobile Applications

By Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Bae, Seungkyu Lee, Choong-Seon Hong. arXiv.org, 2023

649 citations recorded by the source corpus.

Segment Anything Model (SAM) has attracted significant attention due to its impressive zero-shot transfer performance and high versatility for numerous vision applications (like image editing with fine-grained control). Many of such applications need to be run on resource-constraint edge devices, like mobile phones. In this work, we aim to make SAM mobile-friendly by replacing the heavyweight image encoder with a lightweight one. A naive way to train such a new SAM as in the original SAM paper leads to unsatisfactory performance, especially when limited training sources are available. We find that this is mainly caused by the coupled optimization of the image encoder and mask decoder, motivated by which we propose decoupled distillation. Concretely, we distill the knowledge from the heavy image encoder (ViT-H in the original SAM) to a lightweight image encoder, which can be automatically compatible with the mask decoder in the original SAM. The training can be completed on a single GPU within less than one day, and the resulting lightweight SAM is termed MobileSAM which is more than 60 times smaller yet performs on par with the original SAM. For inference speed, With a single GPU, MobileSAM runs around 10ms per image: 8ms on the image encoder and 4ms on the mask decoder. With superior performance, our MobileSAM is around 5 times faster than the concurrent FastSAM and 7 times smaller, making it more suitable for mobile applications. Moreover, we show that MobileSAM can run relatively smoothly on CPU. The code for our project is provided at \href{https://github.com/ChaoningZhang/MobileSAM}{\textcolor{red}{MobileSAM}}), with a demo showing that MobileSAM can run relatively smoothly on CPU.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/fdaa21fce5ee5aadf1bb8c55d5f7c3155b905846

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Segment Anything in High Quality (2023, background): “5% parameters, to extend its capability to high-quality segmentation.”
VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation (2023, methodology): “If a target object instance is detected, we use Mobile-SAM [30] to extract its contour using the RGB image and the detected bounding box.”
Foundation Models Defining a New Era in Vision: A Survey and Outlook (2025, neutral): “Similarly, visual prompt-based models like Segment Anything [11] would beneﬁt from a real-time speed to ensure intractability [150], [192].”
Personalize Segment Anything Model with One Shot (2023, background): “There are some concurrent works extending SAM for high-quality segmentation (Ke et al., 2023), faster inference speed (Zhao et al., 2023; Zhang et al., 2023a), all-purpose matching (Liu et al., 2023), 3D reconstruction (Cen et al., 2023), object tracking (Yang et al., 2023), medical (Ma & Wang,…”
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, methodology): “For instance, in the data labeling process, Grounded SAM can collaborate with the faster inference SAM models, such as FastSAM [85], MobileSAM [76], Light-HQ-SAM [24], and EfficientSAM [63].”
Foundation models in robotics: Applications, challenges, and the future (2023, positive): “FastSAM [60] and MobileSAM [61] achieve comparable performance to SAM at faster inference speeds.”
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything (2023, methodology): “FastSAM[68] develops a CNN-based architecture, YOLOv8-seg[30], to segment all objects in an image for efficiency improvement.”
Rep ViT: Revisiting Mobile CNN From ViT Perspective (2023, methodology): “We first compare our RepViT-SAM with Mobile-SAM [71] and the original SAM [33] with ViT-B image We then evaluate the performance of our RepViT-SAM on zero-shot edge detection using BSDS500 [2, 45], zero-shot instance segmentation using COCO [37], and segmentation in the wild benchmark (SegInW),…”
Gaussian Grouping: Segment and Edit Anything in 3D Scenes (2023, background): “While there has been remarkable progress in 2D scene understanding brought by SAM and its variants [12, 15, 58], their extension to 3D has been constrained.”
Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and Reasoning (2026, unknown): “The generated texts are grounded to obtain masks using Ground-ingDINO [20] and MobileSAM [47].”