A Survey on Deep Learning Technique for Video Segmentation

By Wenguan Wang, Tianfei Zhou, Fatih Murat Porikli, David J. Crandall, Luc Van Gool. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021

304 citations recorded by the source corpus.

Video segmentation—partitioning video frames into multiple segments or objects—plays a critical role in a broad range of practical applications, from enhancing visual effects in movie, to understanding scenes in autonomous driving, to creating virtual background in video conferencing. Recently, with the renaissance of connectionism in computer vision, there has been an influx of deep learning based approaches for video segmentation that have delivered compelling performance. In this survey, we comprehensively review two basic lines of research — generic object segmentation (of unknown categories) in videos, and video semantic segmentation — by introducing their respective task settings, background concepts, perceived need, development history, and main challenges. We also offer a detailed overview of representative literature on both methods and datasets. We further benchmark the reviewed methods on several well-known datasets. Finally, we point out open issues in this field, and suggest opportunities for further research. We also provide a public website to continuously track developments in this fast advancing field: https://github.com/tfzhou/VS-Survey.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4f3ed96bd367930a5f9c46600a3bd3311c263c67

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Transformer-Based Visual Segmentation: A Survey (2023, background): “Meanwhile, there are previous surveys on the deep-learning-based segmentation [37], [38], [39].”
A Generalist Framework for Panoptic Segmentation of Images and Videos (2022, background): “Among the numerous video segmentation tasks, video object segmentation (VOS) [46, 61] is perhaps most canonical task, which entails the segmentation of key objects (of unknown categories).”