PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment

By Kaixin Wang, J. Liew, Yingtian Zou, Daquan Zhou, Jiashi Feng. 2019

1,460 citations recorded by the source corpus.

Despite the great progress made by deep CNNs in image semantic segmentation, they typically require a large number of densely-annotated images for training and are difficult to generalize to unseen object categories. Few-shot segmentation has thus been developed to learn to perform segmentation from only a few annotated examples. In this paper, we tackle the challenging few-shot segmentation problem from a metric learning perspective and present PANet, a novel prototype alignment network to better utilize the information of the support set. Our PANet learns class-specific prototype representations from a few support images within an embedding space and then performs segmentation over the query images through matching each pixel to the learned prototypes. With non-parametric metric learning, PANet offers high-quality prototypes that are representative for each semantic class and meanwhile discriminative for different classes. Moreover, PANet introduces a prototype alignment regularization between support and query. With this, PANet fully exploits knowledge from the support and provides better generalization on few-shot segmentation. Significantly, our model achieves the mIoU score of 48.1% and 55.7% on PASCAL-5i for 1-shot and 5-shot settings respectively, surpassing the state-of-the-art method by 1.8% and 8.6%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Yi Yang
  5. Bernt Schiele
  6. Zeynep Akata
  7. Jiashi Feng
  8. Yoshua Bengio

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Yi Yang
  6. Xiaogang Wang
  7. Jitendra Malik
  8. Bernt Schiele

In Meta & Continual Learning

  1. Fabio Cermelli
  2. Pietro Zanuttigh
  3. Umberto Michieli
  4. Barbara Caputo
  5. Zeynep Akata
  6. Yongqin Xian
  7. Alexander S. Ecker
  8. Timo Lüddecke

Citations in the corpus