Continual Segmentation with Disentangled Objectness Learning and Class Recognition

By Yizheng Gong, Siyue Yu, Xiaoyang Wang, Jimin Xiao. 2024

18 citations recorded by the source corpus.

Most continual segmentation methods tackle the prob-lem as a per-pixel classification task. However, such a paradigm is very challenging, and we find query-based seg-menters with built-in objectness have inherent advantages compared with per-pixel ones, as objectness has strong transfer ability and forgetting resistance. Based on these findings, we propose CoMasTRe by disentangling continual segmentation into two stages: forgetting-resistant continual objectness learning and well-researched continual classi-fication. CoMasTRe uses a two-stage segmenter learning class-agnostic mask proposals at the first stage and leaving recognition to the second stage. During continual learning, a simple but effective distillation is adopted to strengthen objectness. To further mitigate the forgetting of old classes, we design a multi-label class distillation strategy suited for segmentation. We assess the effectiveness of CoMas-TRe on PASCAL VOC and ADE20K. Extensive experiments show that our method outperforms per-pixel and query-based methods on both datasets. Code will be available at https://github.com/jordangong/CoMasTRe.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition