Track Anything: Segment Anything Meets Videos
By Jinyu Yang, Mingqi Gao, Zhe Li, Shanghua Gao, Fang Wang, Fengcai Zheng. arXiv.org, 2023
343 citations recorded by the source corpus.
Recently, the Segment Anything Model (SAM) gains lots of attention rapidly due to its impressive segmentation performance on images. Regarding its strong ability on image segmentation and high interactivity with different prompts, we found that it performs poorly on consistent segmentation in videos. Therefore, in this report, we propose Track Anything Model (TAM), which achieves high-performance interactive tracking and segmentation in videos. To be detailed, given a video sequence, only with very little human participation, i.e., several clicks, people can track anything they are interested in, and get satisfactory results in one-pass inference. Without additional training, such an interactive design performs impressively on video object tracking and segmentation. All resources are available on {https://github.com/gaomingqi/Track-Anything}. We hope this work can facilitate related research.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/c0cf0971c153a84bbf0729d289b3b960969bb5dd
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Li Fei-Fei
- Trevor Darrell
- Jitendra Malik
- Abhinav Kumar Gupta
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
Citations in the corpus
- Foundation models in robotics: Applications, challenges, and the future (2023, positive): “The Track Anything Model (TAM) [62] combines SAM and XMem [165], an advanced video object segmentation (VOS) model, to achieve interactive video object tracking and segmentation.”
- LangSplat: 3D Language Gaussian Splatting (2023, background): “[38] introduced [48], super-resolution [25], image matting [45], object tracking [11, 42], medical image segmentation [29], image editing [12], and so on.”
- Faster Segment Anything: Towards Lightweight SAM for Mobile Applications (2023, background): “Beyond object segmentation, multiple works have also shown its versatility in other fields, including image editing Rombach et al. [2022], as well as inpainting tasks Yu et al. [2023], object tracking within videos Yang et al. [2023], Zxyang [2023].”
- EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything (2023, background): “…shown remarkable zero-shot transfer performance and high versatility for many vision tasks including a variety of segmentation application[7, 8, 10, 17], in-painting[67], image restoration[29], image editing[21], image shadow removal[69], object tracking[14, 65], and 3D object reconstruction[49].”