CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning

By Chi Zhang, Guosheng Lin, Fayao Liu, Rui Yao, Chunhua Shen. 2019

674 citations recorded by the source corpus.

Recent progress in semantic segmentation is driven by deep Convolutional Neural Networks and large-scale labeled image datasets. However, data labeling for pixel-wise segmentation is tedious and costly. Moreover, a trained model can only make predictions within a set of pre-defined classes. In this paper, we present CANet, a class-agnostic segmentation network that performs few-shot segmentation on new classes with only a few annotated images available. Our network consists of a two-branch dense comparison module which performs multi-level feature comparison between the support image and the query image, and an iterative optimization module which iteratively refines the predicted results. Furthermore, we introduce an attention mechanism to effectively fuse information from multiple support examples under the setting of k-shot learning. Experiments on PASCAL VOC 2012 show that our method achieves a mean Intersection-over-Union score of 55.4% for 1-shot segmentation and 57.1% for 5-shot segmentation, outperforming state-of-the-art methods by a large margin of 14.6% and 13.2%, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, methodology): “It’s noted that the performance of few-shot segmentation methods (CANet[48] and ARNet[40]) without updating drops significantly, due to the conflicts between feature representations of new and base classes.”
Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “Apart from image classification, few-shot learning has also been applied to dense prediction tasks [6, 21, 23, 53, 54] and object detection [45].”
Mnemonics Training: Multi-Class Incremental Learning Without Forgetting (2020, background): “Meta-learning [5, 31, 35, 15] is another BOP in which a meta-learner is optimized subject to the optimality of the base-learner.”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “There are different meta strategies, among which the gradient descent based methods are particularly promising for today’s neural networks [1, 13–15, 20, 25, 38, 70, 74, 81, 83, 84, 86].”
ARNet: Active-Reference Network for Few-Shot Image Semantic Segmentation (2021, methodology): “Zhang et al. [8] tries to model such process as a learnable structure by utilizing an attention mechanism for result fusion.”
Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (2022, methodology): “It represents the “expand & concatenate” operations [66] in our work.”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, methodology): “During training, four data augmentation strategies including normalization, horizontal ﬂipping, random cropping and random resizing are used [20].”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, methodology): “However, given only a few annotated training examples, the problem of poor generalization ability of the deep networks is yet the primary concern that many few-shot segmentation methods [10, 12, 13, 19, 33, 36, 37, 46, 54, 61, 63, 69, 70, 74, 75, 80, 83, 86, 87, 89] struggle to address.”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “…task, we ﬁrst train on base classes and then simply operate a balanced ﬁne-tuning with both novel and base classes, which is simpler and more direct than explicit feature-comparison approaches used in other class-agnostic few-shot methods [11], [12], towards segmentation or object counting.”
Progressive One-shot Human Parsing (2020, positive): “Following the one/few-shot learning (Koch, Zemel, and Salakhutdinov 2015; Finn, Abbeel, and Levine 2017; Snell, Swersky, and Zemel 2017; Sung et al. 2018; Chen et al. 2020; Liu et al. 2020a; Tian et al. 2020b), a typical OS3 solution is to learn a good metric (Zhang et al. 2018; Kate et al. 2018; Zh”
A Few Guidelines for Incremental Few-Shot Segmentation (2020, methodology): “Few-Shot Semantic Segmentation [36, 33, 9, 41, 48, 38] deals with how to learn to segment an image from a scarce training set.”
Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation (2020, methodology): “In PANet [3] and CANet [5], support and query branches share the encoder network, and the foreground prototype is computed using global average pooling on the foreground features.”
Generalized Few-shot Semantic Segmentation (2020, background): “They are CANet [58], PFENet [39] SCL [55] and PANet [43]).”
FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation (2019, result): “More importantly, our model trained only on FSS-1000 without fine-tuning on PASCAL-5 (Ours*) achieves much better results compared to models trained on PASCAL-5 (Ours), exceeding the state-of-the-art performance of the very recent [39, 36].”
Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation (2019, methodology): “[26] trains an additional branch to weight the k global descriptors before averaging them.”
DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers (2020, neutral): “Besides image classification, few-shot learning is also investigated in image segmentation [34, 74, 75] and object detection tasks [72].”