Diffusion Models for Implicit Image Segmentation Ensembles

By Julia Wolleb, Robin Sandkühler, Florentin Bieder, Philippe Valmaggia, Philippe C. Cattin. 2021

422 citations recorded by the source corpus.

Diffusion models have shown impressive performance for generative modelling of images. In this paper, we present a novel semantic segmentation method based on diffusion models. By modifying the training and sampling scheme, we show that diffusion models can perform lesion segmentation of medical images. To generate an image specific segmentation, we train the model on the ground truth segmentation, and use the image as a prior during training and in every step during the sampling process. With the given stochastic sampling process, we can generate a distribution of segmentation masks. This property allows us to compute pixel-wise uncertainty maps of the segmentation, and allows an implicit ensemble of segmentations that increases the segmentation performance. We evaluate our method on the BRATS2020 dataset for brain tumor segmentation. Compared to state-of-the-art segmentation models, our approach yields good segmentation results and, additionally, detailed uncertainty maps.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Elucidating the Design Space of Diffusion-Based Generative Models (2022, background): “They are also rapidly finding use in other domains such as audio [27, 37] and video [19] generation, image segmentation [4, 54] and language translation [34].”
DiffusionDet: Diffusion Model for Object Detection (2022, methodology): “Some pioneer works tried to adopt the diffusion model for image segmentation tasks [1, 5, 6, 13, 31, 47, 97], for example, Chen et al .”
Imagic: Text-Based Real Image Editing with Diffusion Models (2022, background): “Diffusion models [22, 58, 60, 66] are a family of generative models that has recently gained traction, as they advanced the state-of-the-art in image generation [16, 31, 61, 65], and have been deployed in various downstream applications such as image restoration [30, 52], adversarial purification [1”
MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model (2022, methodology): “Comparing against DPM-based model proposed specifically for the brain tumor segmentation, i.e., EnsemDiff, it improves 2.3% on Dice and 2.4% on IoU, which indicates the effectiveness of our unique techniques, i.e, dynamic conditioning and FF-Parser.”
Diffusion models in medical imaging: A comprehensive survey (2022, positive): “Algorithm Networks Core Ideas Practical Use Cases Denoising Diffusion Probabilistic Models (DDPMs) (1)AnoDDPM [69] (2)CDPM [70] (3)IITM-Diffusion [71] (4)AnoDDIM [16] (5)PET-DDPM [63] (6)DenoOCT-DDPM [64] (7)brainSPADE [60] (8)DARL [68] (9)IISE [62] (10)*MT-Diffusion [49] (11)SynDiff [51] (12)MC-DDP”
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation (2023, positive): “We utilized the BTCV dataset (Fang and Yan 2020), a widely-used and publicly available benchmark with twelve anatomies as the benchmark.”
MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer (2023, methodology): “Inspired by its success, some researchers have attempted to apply DPM in the field of medical image segmentation [6, 13,16,23, 25].”