Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation

By Zilong Huang, Wentian Hao, Xinggang Wang, Jianqiang Huang, Wenyu Liu, Xiansheng Hua. 2021

13 citations recorded by the source corpus.

Despite their success for semantic segmentation, convolutional neural networks are ill-equipped for incremental learning, \ie, adapting the original segmentation model as new classes are available but the initial training data is not retained. Actually, they are vulnerable to catastrophic forgetting problem. We try to address this issue by"inverting"the trained segmentation network to synthesize input images starting from random noise. To avoid setting detailed pixel-wise segmentation maps as the supervision manually, we propose the SegInversion to synthesize images using the image-level labels. To increase the diversity of synthetic images, the Scale-Aware Aggregation module is integrated into SegInversion for controlling the scale (the number of pixels) of synthetic objects. Along with real images of new classes, the synthesized images will be fed into the distillation-based framework to train the new segmentation model which retains the information about previously learned classes, whilst updating the current model to learn the new ones. The proposed method significantly outperforms other incremental learning methods and obtains state-of-the-art performance on the PASCAL VOC 2012 and ADE20K datasets. The code and models will be made publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Representation Compensation Networks for Continual Semantic Segmentation (2022, positive): “In this field, continual class segmentation is a classic setting, with great progress made by several previous work: [41,92] explore rehearsal-based methods to review old knowledge; MiB [8] models the potential classes to solve the ambiguous of background class; PLOP [27] applies knowledge distillat”
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation (2021, methodology): “[73] proposed after our PLOP a pseudo-rehearsal method for segmentation that while effective, incurs an important time overhead (up to 3× longer).”
Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (2022, background): “Contemporary works [16,21,39] have developed several replay-based methods for CSS.”