CRNet: Cross-Reference Networks for Few-Shot Segmentation

By Weide Liu, Chi Zhang, Guosheng Lin, Fayao Liu. 2020

240 citations recorded by the source corpus.

Over the past few years, state-of-the-art image segmentation algorithms are based on deep convolutional neural networks. To render a deep network with the ability to understand a concept, humans need to collect a large amount of pixel-level annotated data to train the models, which is time-consuming and tedious. Recently, few-shot segmentation is proposed to solve this problem. Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. In this paper, we propose a cross-reference network (CRNet) for few-shot segmentation. Unlike previous works which only predict the mask in the query image, our proposed model concurrently makes predictions for both the support image and the query image. With a cross-reference mechanism, our network can better find the co-occurrent objects in two images, thus helping the few-shot segmentation task. We also develop a mask refinement module to recurrently refine the prediction of the foreground regions. For the k-shot learning, we propose to finetune parts of networks to take advantage of multiple labeled support images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Guosheng Lin
2. Chi Zhang
3. Chunhua Shen
4. Tong Lu
5. Weide Liu
6. Qixiang Ye
7. Jianbin Jiao
8. Jing Han

In Computer Vision and Pattern Recognition

1. Chunhua Shen
2. Luc Van Gool
3. Guosheng Lin
4. Dacheng Tao
5. Chi Zhang
6. Qixiang Ye
7. Xiaogang Wang
8. Yi Yang

Citations in the corpus

• Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, background): “Few-shot semantic segmentation [21, 28, 40] addresses to segment a new category of images with few samples, decreasing the cost of expensive pixel-level annotations.”
• Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “Apart from image classification, few-shot learning has also been applied to dense prediction tasks [6, 21, 23, 53, 54] and object detection [45].”
• DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers (2020, methodology): “Besides image classification, fewshot learning is also investigated in other computer vision tasks, such as image segmentation [34, 73, 74].”
• Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “CRNet [22] proposed a cross-reference mechanism to concurrently make predictions for both the support image(s) and the query image, enforcing co-occurrence of objects and thereby improving the semantic transfer.”
• Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, background): “In the past few years, Deep Neural Networks have made remarkable breakthroughs in many computer vision tasks, such as image classification [9, 49, 63], object detection [15] and semantic segmentation [7, 34, 35, 37, 64, 65].”
• Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “However, given only a few annotated training examples, the problem of poor generalization ability of the deep networks is yet the primary concern that many few-shot segmentation methods [10, 12, 13, 19, 33, 36, 37, 46, 54, 61, 63, 69, 70, 74, 75, 80, 83, 86, 87, 89] struggle to address.”
• CycleSegNet: Object Co-Segmentation With Cycle Refinement and Region Correspondence (2021, methodology): “Image cosegmentation algorithms have shown their usages in various computer vision tasks, such as image retrieval [38], 3D reconstruction [29], photo collections [33], image matching [6], [45], [46] and video object tracking and segmentation [23]–[25], [33].”
• Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? (2020, background): “Few-shot segmentation, which has gained popularity recently [10, 17, 19, 23, 25, 33, 36, 37, 38, 39, 41, 42], is no exception.”
• ARNet: Active-Reference Network for Few-Shot Image Semantic Segmentation (2021, methodology): “To better find the co-occurrence objects, CRNet [3] proposes a cross-referencer mechanism to concurrently make predictions for both the support and query images, where we show its workflow in Fig.”
• DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning (2020, background): “Few-shot learning has also been investigated for other computer vision tasks, such as image segmentation [92], [93], [94] and object detection [95].”
• Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation (2020, methodology): “In our work, we first examine a collection of few-shot learning techniques in images as baselines for point cloud data, such as metric-based methods [2, 3] and relation-based methods [4–6], which are two dominant streams in few-shot image segmentation.”