Extending pretrained segmentation networks with additional anatomical structures

By Firat Özdemir, Orçun Göksel. 2018

48 citations recorded by the source corpus.

PurposeFor comprehensive surgical planning with sophisticated patient-specific models, all relevant anatomical structures need to be segmented. This could be achieved using deep neural networks given sufficiently many annotated samples; however, datasets of multiple annotated structures are often unavailable in practice and costly to procure. Therefore, being able to build segmentation models with datasets from different studies and centers in an incremental fashion is highly desirable.MethodsWe propose a class-incremental framework for extending a deep segmentation network to new anatomical structures using a minimal incremental annotation set. Through distilling knowledge from the current network state, we overcome the need for a full retraining.ResultsWe evaluate our methods on 100 MR volumes from SKI10 challenge with varying incremental annotation ratios. For 50% incremental annotations, our proposed method suffers less than 1% Dice score loss in retaining old-class performance, as opposed to 25% loss of conventional finetuning. Our framework inherently exploits transferable knowledge from previously trained structures to incremental tasks, demonstrated by results superior even to non-incremental training: In a single volume one-shot incremental learning setting, our method outperforms vanilla network performance by>11% in Dice.ConclusionsWith the presented method, new anatomical structures can be learned while retaining performance for older structures, without a major increase in complexity and memory footprint, hence suitable for lifelong class-incremental learning. By leveraging information from older examples, a fraction of annotations can be sufficient for incrementally building comprehensive segmentation models. With our meta-method, a deep segmentation network is extended with only a minor addition per structure, thus can be applicable also for future network architectures.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Medical Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Medical Imaging

Citations in the corpus

Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, methodology): “To the best of our knowledge, the problem of incremental learning in semantic segmentation has been addressed only in [19], [20], [22], [47].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “Replay has also been used to perform continual semantic segmentation of medical images (Ozdemir, Fuernstahl, & Goksel, 2018; Ozdemir & Goksel, 2019), remote sensing data (Tasar, Tarabalka, & Alliez, 2019; Wu, Wang et al., 2019), and on standard computer vision benchmarks (Cermelli, Mancini, Bulo,…”
Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “Hence, it emerged the need for continual approaches specifically targeted to solve the semantic segmentation task [47, 58, 42, 43, 33, 4].”
PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, background): “This setup, referred here as Continual Semantic Segmentation (CSS), has emerged very recently for medical applications [51, 52] before being proposed for general segmentation datasets [49, 7].”
Modeling the Background for Incremental Learning in Semantic Segmentation (2020, methodology): “[26, 27] describe an ICL approach for medical imaging, extending a standard image-level classiﬁcation method [18] to segmentation and devising a strategy to select relevant samples of old datasets for rehearsal.”
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation (2021, background): “This setup, referred here as Continual Semantic Segmentation (CSS), has emerged very recently for specialized applications [11], [12], [13] before being proposed for general segmentation datasets [14], [15], [16].”
Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery (2021, methodology): “However, like the continual learning method of semantic segmentation applied to medical images [50], [51] and general images [33]–[35], KD-BCE considers only the class continual learning setting, that is, both the new class and the old class come from training data with the same distribution.”
A Few Guidelines for Incremental Few-Shot Segmentation (2020, background): “This problem has been extensively studied in image classiﬁcation [23, 21, 4, 34, 17, 17, 45], and only recently in semantic segmentation [30, 28, 3].”
Continual Segmentation with Disentangled Objectness Learning and Class Recognition (2024, unknown): “…continual learning shines in dense prediction tasks, such as semantic segmentation [3, 5, 12], since the annotations are laborious to obtain, and sometimes the learned samples are inaccessible, such as the nature of stream data in autonomous driving [34] or patient privacy in medical imaging [35].”
Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (2022, background): “Research on continual learning for semantic segmentation (CSS) only emerged recently in medical imaging [27, 28] and general scene understanding [2, 9].”
Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation (2021, unknown): “There are few works [27, 31, 32, 41, 5] that focus on the problem of incremental learning in semantic segmentation.”