Video Instance Segmentation Tracking With a Modified VAE Architecture
By Chung-Ching Lin, Ying Hung, Rogério Schmidt Feris, Linglin He. 2020
57 citations recorded by the source corpus.
We propose a modified variational autoencoder (VAE) architecture built on top of Mask R-CNN for instance-level video segmentation and tracking. The method builds a shared encoder and three parallel decoders, yielding three disjoint branches for predictions of future frames, object detection boxes, and instance segmentation masks. To effectively solve multiple learning tasks, we introduce a Gaussian Process model to enhance the statistical representation of VAE by relaxing the prior strong independent and identically distributed (iid) assumption of conventional VAEs and allowing potential correlations among extracted latent variables. The network learns embedded spatial interdependence and motion continuity in video data and creates a representation that is effective to produce high-quality segmentation masks and track multiple instances in diverse and unstructured videos. Evaluation on a variety of recently introduced datasets shows that our model outperforms previous methods and achieves the new best in class performance.
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Lei Zhang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
In Generative Models
- David J. Fleet
- Ting-Li Chen
- Geoffrey E. Hinton
- Saurabh Saxena
- Lala Li
- Lei Zhang
- Yoshua Bengio
- Ruslan Salakhutdinov
Citations in the corpus
- Crossover Learning for Fast Online Video Instance Segmentation (2021, unknown): “[35] introduces a modified variational auto-encoder to solve the VIS task.”
- SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation (2021, methodology): “Majority of VIS methods [2, 8, 11, 16, 19, 25, 26] follow the two-stage paradigm based on Mask R-CNN.”
- A Survey on Deep Learning Technique for Video Segmentation (2021, unknown): “[230], [232], [235], [236], [237], [252].”
- Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation (2021, methodology): “Besides, [26] proposes a modified variation autoencoder (VAE) architecture built on the top of Mask R-CNN [18].”
- A Generalist Framework for Panoptic Segmentation of Images and Videos (2022, methodology): “And like image segmentation, most existing methods are specialist models comprising multiple stages with pipe-lined frameworks, e.g., track-detect [67, 57, 36, 6, 44], clip-match [3, 5], propose-reduce [37].”