Learning Video Object Segmentation From Unlabeled Videos

By Xiankai Lu, Wenguan Wang, Jianbing Shen, Yu-Wing Tai, David J. Crandall, Steven C. H. Hoi. 2020

154 citations recorded by the source corpus.

We propose a new method for video object segmentation (VOS) that addresses object pattern learning from unlabeled videos, unlike most existing methods which rely heavily on extensive annotated data. We introduce a unified unsupervised/weakly supervised learning framework, called MuG, that comprehensively captures intrinsic properties of VOS at multiple granularities. Our approach can help advance understanding of visual patterns in VOS and significantly reduce annotation burden. With a carefully-designed architecture and strong representation learning ability, our learned model can be applied to diverse VOS settings, including object-level zero-shot VOS, instance-level zero-shot VOS, and one-shot VOS. Experiments demonstrate promising performance in these settings, as well as the potential of MuG in leveraging unlabeled data to further improve the segmentation accuracy.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition (2021, methodology): “The ORBIT dataset can be used to explore a wide set of real-world recognition tasks from continual learning [27, 28] to video segmentation [25, 34, 29].”
A Survey on Deep Learning Technique for Video Segmentation (2021, methodology): “Each of the annotated object tracklets will be matched with one of predicted tracklets according to TABLE 7 Quantitative Object-Level AVOS Results on DAVIS16 [17] val (Section 5.1.2) in Terms of Region Similarity J , Boundary Accuracy F and Time Stability T Method J F T FPS" mean" recall" decay# mea”
Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, background): “Few shot learning has also been explored in video analysis [20, 21].”
Video Object Segmentation with Episodic Graph Memory Networks (2020, background): “, the more advanced, is the matching based strategy [7, 11, 15, 17, 25, 31, 49, 51, 69, 70] which usually trains a prototypical Siamese matching network to find the most matching pixel (or embedding in the feature space) between the first frame (or a segmented frame) and the query frame, and then ac”
Learning About Objects by Learning to Interact with Them (2020, background): “[38] propose a multi-granular approach to find objects in videos.”