Learn the new, keep the old: Extending pretrained models with new anatomy and images

By Firat Özdemir, Philipp Fürnstahl, Orçun Göksel. 2018

49 citations recorded by the source corpus.

Deep learning has been widely accepted as a promising solution for medical image segmentation, given a sufficiently large representative dataset of images with corresponding annotations. With ever increasing amounts of annotated medical datasets, it is infeasible to train a learning method always with all data from scratch. This is also doomed to hit computational limits, e.g., memory or runtime feasible for training. Incremental learning can be a potential solution, where new information (images or anatomy) is introduced iteratively. Nevertheless, for the preservation of the collective information, it is essential to keep some "important" (i.e. representative) images and annotations from the past, while adding new information. In this paper, we introduce a framework for applying incremental learning for segmentation and propose novel methods for selecting representative data therein. We comparatively evaluate our methods in different scenarios using MR images and validate the increased learning capacity with using our methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, background): “Recently, due to the importance of it for several high level tasks, some approaches have proposed to continuously update a deep network for computing semantic segmentation maps [19], [20], [21], [22], [23].”
An EM Framework for Online Incremental Learning of Semantic Segmentation (2021, neutral): “To address this limitation, several recent works start to explore the problem of class-incremental semantic segmentation [5, 23, 24, 30].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “Replay has also been used to perform continual semantic segmentation of medical images (Ozdemir, Fuernstahl, & Goksel, 2018; Ozdemir & Goksel, 2019), remote sensing data (Tasar, Tarabalka, & Alliez, 2019; Wu, Wang et al., 2019), and on standard computer vision benchmarks (Cermelli, Mancini, Bulo,…”
Self-Training for Class-Incremental Semantic Segmentation (2020, background): “It is also investigated in remote sensing [9] and medical data [47] for incremental semantic segmentation.”
PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, background): “This setup, referred here as Continual Semantic Segmentation (CSS), has emerged very recently for medical applications [51, 52] before being proposed for general segmentation datasets [49, 7].”
Modeling the Background for Incremental Learning in Semantic Segmentation (2020, methodology): “[24, 25] describe an ICL approach for medical imaging, extending a standard image-level classification method [18] to segmentation and devising a strategy to select relevant samples of old datasets for rehearsal.”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, methodology): “Incremental semantic segmentation has also been applied to remote sensing [93] and medical data [94].”
Extending pretrained segmentation networks with additional anatomical structures (2018, background): “In order to account for both of these conditions (i) and (ii), we propose to first pick a large batch of samples for which the trained model is “confident” [9] and then to prune them with the aim of maximizing how well they represent the entire dataset [4].”
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation (2021, background): “This setup, referred here as Continual Semantic Segmentation (CSS), has emerged very recently for specialized applications [11], [12], [13] before being proposed for general segmentation datasets [14], [15], [16].”
Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation (2021, unknown): “There are few works [27, 31, 32, 41, 5] that focus on the problem of incremental learning in semantic segmentation.”
Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery (2021, methodology): “However, like the continual learning method of semantic segmentation applied to medical images [50], [51] and general images [33]–[35], KD-BCE considers only the class continual learning setting, that is, both the new class and the old class come from training data with the same distribution.”
Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (2022, background): “Research on continual learning for semantic segmentation (CSS) only emerged recently in medical imaging [27, 28] and general scene understanding [2, 9].”