Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

By Wenguan Wang, Xiankai Lu, Jianbing Shen, David J. Crandall, Ling Shao. IEEE International Conference on Computer Vision, 2019

294 citations recorded by the source corpus.

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-of-the-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8e16fd9bace78ac87b9c35a7fbaf5d725cd35b53

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Video Object Segmentation From Unlabeled Videos (2020, background): “Recent advances have been driven by deep learning techniques, from early, relatively simple architectures, such as recurrent network[45, 32, 63], and two-stream network [6, 49, 77], to recent, more powerful designs, such as teacher-student adaption [44], neural co-attention [26] and graph neural net”
CycleSegNet: Object Co-Segmentation With Cycle Refinement and Region Correspondence (2021, unknown): “This method can make use of all images in the group to make predictions for each image. b) Following [66], we randomly divide all images into N/k small groups and each group has k images.”
JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (2020, background): “Many SOD models have been developed under the assumption that the inputs are individual RGB/color images [21, 47, 66, 74–76] or sequences [56, 62, 67, 68].”
Video Object Segmentation with Episodic Graph Memory Networks (2020, methodology): “1 79.2 73.8 83.4 80.6 Decay# 5.1 11.4 3.4 2.9 5.1 2.5 1.8 1.3 0.7 T Mean # 54.8 65.4 58.2 60.6 60.3 71.0 65.4 66.7 39.0 ARP PDB MotAdapt LSMO AGS COSNet AGNN AnDi Method [19] [42] [41] [46] [55] [24] [52] [68] Ours J Mean " 76.2 77.2 77.2 78.2 79.7 80.5 80.7 81.7 82.5 Recall" 89.1 91.1 93.”
Learning About Objects by Learning to Interact with Them (2020, background): “[55, 64, 60] require a video at test time to find the moving objects.”
STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos (2020, methodology): “s and optical ow cues. In contrast to ours, this method processes each frame separately, employs K-means clustering for object localization, and cannot separate dierent object instances. Wang et al. [84] employ an attentive graph neural network [25] and use dierentiable message passing to propagate ”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, background): “ool in many vision-language applications and sequential modeling tasks, such as visual question answering [39,42,76,79], visual dialog [74,85], vision-language navigation [66], and video segmentation [40,60], showing its eectiveness in capturing the underlying relations between dierent entities. Ins”