Self-Training for Class-Incremental Semantic Segmentation

By Lu Yu, Xialei Liu, Joost van de Weijer. 2020

70 citations recorded by the source corpus.

In class-incremental semantic segmentation, we have no access to the labeled data of previous tasks. Therefore, when incrementally learning new classes, deep neural networks suffer from catastrophic forgetting of previously learned knowledge. To address this problem, we propose to apply a self-training approach that leverages unlabeled data, which is used for rehearsal of previous knowledge. Specifically, we first learn a temporary model for the current task, and then, pseudo labels for the unlabeled data are computed by fusing information from the old model of the previous task and the current temporary model. In addition, conflict reduction is proposed to resolve the conflicts of pseudo labels generated from both the old and temporary models. We show that maximizing self-entropy can further improve results by smoothing the overconfident predictions. Interestingly, in the experiments, we show that the auxiliary data can be different from the training data and that even general-purpose, but diverse auxiliary data can lead to large performance gains. The experiments demonstrate the state-of-the-art results: obtaining a relative gain of up to 114% on Pascal-VOC 2012 and 8.5% on the more challenging ADE20K compared to previous state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “PLOP [292], RECALL [293], SSUL [294], Self-Training [295] and WILSON [296] explicitly apply the old model to generate pseudo-labels of the old classes.”
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation (2021, methodology): “Moreover, we compare with [74], where an unlabeled external dataset such as COCO [112] is used through pseudo-labelling to improve performance on Pascal-VOC, as both datasets present significant overlap in terms of classes and domains.”
Endpoints Weight Fusion for Class Incremental Semantic Segmentation (2023, methodology): “It is worth noting that there are several methods [6, 34, 49] that introduce different forms of auxiliary data to assist continual semantic segmentation, helping the model build better pseudo-labels or enhance memory for old knowledge.”
Class-Incremental Learning: A Survey (2023, background): “Besides, works also investigate the application of generative replay [160] and auxiliary data [160], [280] for data replay.”
Continual Semantic Segmentation Leveraging Image-level Labels and Rehearsal (2022, background): “For instance, [Yu et al. , 2020] assumed access to ∼ 120K and ∼ 1.8M additional unlabelled images with classes that overlap those of the main dataset, which is a strong assumption that cannot hold in many real applications.”
Continual Learning of Semantic Segmentation using Complementary 2D-3D Data Representations (2021, background): “A limited number of works has explored semantic segmentation in continual learning [20, 30, 6, 10, 9, 19].”