Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation

By Chi Zhang, Guosheng Lin, Fayao Liu, Jiushuang Guo, Qingyao Wu, Rui Yao. 2019

358 citations recorded by the source corpus.

One-shot image segmentation aims to undertake the segmentation task of a novel class with only one training image available. The difficulty lies in that image segmentation has structured data representations, which yields a many-to-many message passing problem. Previous methods often simplify it to a one-to-many problem by squeezing support data to a global descriptor. However, a mixed global representation drops the data structure and information of individual elements. In this paper, we propose to model structured segmentation data with graphs and apply attentive graph reasoning to propagate label information from support data to query data. The graph attention mechanism could establish the element-to-element correspondence across structured data by learning attention weights between connected graph nodes. To capture correspondence at different semantic levels, we further propose a pyramid-like structure that models different sizes of image regions as graph nodes and undertakes graph reasoning at different levels. Experiments on PASCAL VOC 2012 dataset demonstrate that our proposed network significantly outperforms the baseline method and leads to new state-of-the-art performance on 1-shot and 5-shot segmentation benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Graph Learning

In Meta & Continual Learning

Citations in the corpus

Language-driven Semantic Segmentation (2022, background): “Few-shot approaches (Shaban et al., 2017; Rakelly et al., 2018; Siam et al., 2019; Wang et al., 2019; Zhang et al., 2019; Nguyen & Todorovic, 2019; Liu et al., 2020b; Wang et al., 2020; Tian et al., 2020; Boudiaf et al., 2021; Min et al., 2021) offer ways to learn to segment novel classes based on…”
Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “Apart from image classification, few-shot learning has also been applied to dense prediction tasks [6, 21, 23, 53, 54] and object detection [45].”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “There are different meta strategies, among which the gradient descent based methods are particularly promising for today’s neural networks [1, 13–15, 20, 25, 38, 70, 74, 81, 83, 84, 86].”
Part-aware Prototype Network for Few-shot Semantic Segmentation (2020, methodology): “e the holistic objects representation with query features in each spatial position and introduce a dense comparison module to estimate their prediction.The subsequent method, proposed by Zhang et al. [35], attends to foreground features for each query feature with a graph attention mechanism. These ”
Prior Guided Feature Enrichment Network for Few-Shot Segmentation (2020, positive): “We also compare the Graph Attention Unit (GAU) used in the recent state-of-the-art few-shot segmentation method PGNet [53] to refine the query feature.”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “There have been some attempts [71, 82] in utilizing dense correlations with multilevel features, but they are yet limited in the sense that they simply employ the dense correlations for graph attention, using only a small fraction of intermediate conv layers.”
Few-Shot Segmentation via Cycle-Consistent Transformer (2021, methodology): “Rather than using prototypes to abstract the support information, [37, 30] (Figure 1(c)) propose to employ the attention mechanism to extract information from support foreground pixels for segmenting query.”
Generalized Few-shot Semantic Segmentation (2020, background): “Few-Shot Segmentation (FSSeg) places semantic segmentation in the few-shot scenario [2,8,11,15,18,20,21,24,27,28,33,34,38,41,48,51,52,57, 61, 62, 66, 67], where dense pixel labeling is performed on new classes with only a few support samples.”
CRNet: Cross-Reference Networks for Few-Shot Segmentation (2020, neutral): “oU IoU OSLM [17] VGG16 43.9 61.5 co-fcn [14] VGG16 41.4 60.2 sg-one [27] VGG16 47.1 65.9 R-DFCN [18] VGG16 45.3 66.0 PL [5] VGG16 - 62.3 A-MCG [6] ResNet-50 - 62.2 CANet [26] ResNet-50 57.1 69.6 PGNet[25] ResNet50 58.5 70.5 CRNet VGG16 58.5 71.0 CRNet ResNet50 58.8 71.5 Table 6. Comparison with the ”
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (2021, background): “PGNet, BriNet and DAN [39, 36, 31] introduced dense pixel-to-pixel connections between support and query features to maintain spatial information.”
Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation (2020, methodology): “• Compositional Prototypes (CP): PGNet [6] proposes Graph Attention Unit (GAU), which uses Graph Networks to compute attentive graph based on Inner-Product of each query point to obtain more accurate prototype for each query point.”
Image Segmentation Using Text and Image Prompts (2021, background): “Later works introduce more complex mechanisms to handle one-shot segmentation: The pyramid graph network (PGNet) [34] generates a set of differently-shaped feature maps obtained through adaptive pooling and processes them by individual graph attention units and passed through an atrous spatial…”
DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning (2020, methodology): “,12,14,20, 27,28,37,45,49,57,60,61,65,69–71,75,76]. Other related topics. Besides image classiﬁcation, fewshot learning is also investigated in other computer vision tasks, such as image segmentation [34,73,74]. There are also some previous works related to the techniques adopted in this paper. For ”
Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “We adopt both the single-scale and multiscale [43, 42, 22] evaluation strategies during testing.”
Graph Attention Tracking (2020, positive): “Aiming at solving these issues, we leverage graph attention networks [28, 34] to design an part-to-part information embedding network for object tracking.”
Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, methodology): “Inspired by this, few-shot semantic segmentation has recently received growing interest in the computer vision community [4, 28, 39].”