Incremental Learning Techniques for Semantic Segmentation

By Umberto Michieli, Pietro Zanuttigh. 2019

289 citations recorded by the source corpus.

Deep learning architectures exhibit a critical drop of performance due to catastrophic forgetting when they are required to incrementally learn new tasks. Contemporary incremental learning frameworks focus on image classification and object detection while in this work we formally introduce the incremental learning problem for semantic segmentation in which a pixel-wise labeling is considered. To tackle this task we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones. We propose various approaches working both on the output logits and on intermediate features. In opposition to some recent frameworks, we do not store any image from previously learned classes and only the last model is needed to preserve high accuracy on these classes. The experimental evaluation on the Pascal VOC2012 dataset shows the effectiveness of the proposed approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “full annotations of both the old and new classes in continual learning [301], [303].”
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, methodology): “To the best of our knowledge, the problem of incremental learning in semantic segmentation has been addressed only in [19], [20], [22], [47].”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “4, experimental results demonstrate that our method achieves higher mIoU than previous methods [8, 27, 62] in all three settings.”
Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, background): “Although class-incremental learning has been studied in several computer vision tasks (Li and Hoiem 2017; Shmelkov, Schmid, and Alahari 2017; Michieli and Zanuttigh 2019; Dong et al. 2021), especially image classification, classincremental learning of 3D object detection remains unexplored.”
Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “We report eight methods that represent the current state-of-the-art for incremental learning using pixel-wise supervision: LWF [38], LWF-MC [54], ILT [45], MiB [8], PLOP [21], CIL [33], SDR [46], and RECALL [43].”
Class-Incremental Learning for Action Recognition in Videos (2021, neutral): “Although class-incremental learning has been studied for image classiﬁcation, the research is also active for other applications, including person re-identiﬁcation [42], 3D object classiﬁcation [5], object detection [33], and semantic segmentation [24].”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “To solve this problem, they proposed to separate parameters into global and task-specific parameters, whereby each client can obtain a weighted combination of the task-specific parameters of the other clients for knowledge transfer.”
RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, methodology): “However, using another semantic segmentation dataset raises some concerns about the fact that the pre-training data could contain information about the tasks to be learned in the incremental steps, thus following previous works [6, 7, 1] we decided for a more conservative approach in the main paper ”
An EM Framework for Online Incremental Learning of Semantic Segmentation (2021, neutral): “It is worth noting that most existing approaches focus on tackling the incremental semantic segmentation with no memory of historical training data and in an offlinemanner [5, 23].”
AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training (2021, methodology): “To compare with popular meth-ods on CCL, we use the baselines as suggested in [30], including Elastic Weight Consolidation (EWC) [23], Path Integral (PI) [42], Riemannian Walks (RW) [4], Learning without Forgetting (LwF) [25], LwF-MC [34], ILT[30], and MiB [2].”
SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning (2021, methodology): “The CIL framework has been extended to more complex semantic segmentation tasks [24, 3, 7], motivated by the practical need in various applications such as autonomous driving.”
Self-Supervised Training Enhances Online Continual Learning (2021, background): “, continual object detection [1, 62], continual semantic segmentation [11, 48], or continual learning for robotics applications [23, 37].”
Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, methodology): “RGB label FT LwF [2] ILT [5] CIL [3] MiB [1] SDR (ours) offline Figure 2.”
PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, methodology): “This setup, referred here as Continual Semantic Segmentation (CSS), has emerged very recently for medical applications [51, 52] before being proposed for general segmentation datasets [49, 7].”
Generative Feature Replay For Class-Incremental Learning (2020, methodology): “Note that feature distillation has also been used in other applications [32, 47, 55].”
Incremental Object Detection via Meta-Learning (2020, neutral): “In comparison to the incremental setting for image-classification [14], [15], [16], [17], class-incremental object detection and class-incremental semantic segmentation [18], [19] has been relatively less explored.”
Modeling the Background for Incremental Learning in Semantic Segmentation (2020, methodology): “Moreover, we are the first to provide a comprehensive experimental evaluation of state of the art ICL methods on commonly used semantic segmentation benchmarks and to explicitly introduce and tackle the semantic shift of the background class, a problem recognized but largely overseen by previous wor”
Knowledge Distillation for Incremental Learning in Semantic Segmentation (2019, methodology): “Following the experimental scenarios presented in [1, 13], we start by analyzing the addition of a single class, in alphabetical or frequency-based order, and then move to the addition of 5 and 10 classes, either all together or sequentially.”