EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything

By Yunyang Xiong, Bala Varadarajan, Lemeng Wu, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest N. Iandola and 2 more. Computer Vision and Pattern Recognition, 2023

303 citations recorded by the source corpus.

Segment Anything Model (SAM) has emerged as a powerful tool for numerous vision applications. A key component that drives the impressive performance for zero-shot trans-fer and high versatility is a super large Transformer model trained on the extensive high-quality SA -1 B dataset. While beneficial, the huge computation cost of SAM model has limited its applications to wider real-world applications. To address this limitation, we propose EfficientSAMs, light-weight SAM models that exhibits decent performance with largely reduced complexity. Our idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning. Further, we take SAMI-pretrained light-weight image encoders and mask decoder to build Effi-cientSAMs, and finetune the models on SA -1B for segment anything task. We perform evaluations on multiple vision tasks including image classification, object detection, in-stance segmentation, and semantic segmentation, and find that our proposed pretraining method, SAMI, consistently outperforms other masked image pretraining methods. On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e.g., rv4 AP on COCOILVIS) over other fast SAM models. Our EfficientSAM code and models are available at here.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/30ee82b45277ff528e8419ee9543cb7e27d09311

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Feng Li
2. Hao Zhang
3. Shilong Liu
4. Yukang Chen
5. Lei Zhang
6. Jianwei Yang
7. Yu Liu
8. Guanglu Song

Citations in the corpus

• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, methodology): “For instance, in the data labeling process, Grounded SAM can collaborate with the faster inference SAM models, such as FastSAM [85], MobileSAM [76], Light-HQ-SAM [24], and EfficientSAM [63].”