Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

By Yang Song, Liyue Shen, Lei Xing, Stefano Ermon. International Conference on Learning Representations, 2021

774 citations recorded by the source corpus.

Reconstructing medical images from partial measurements is an important inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing solutions based on machine learning typically train a model to directly map measurements to medical images, leveraging a training dataset of paired images and measurements. These measurements are typically synthesized from images using a fixed physical model of the measurement process, which hinders the generalization capability of models to unknown measurement processes. To address this issue, we propose a fully unsupervised technique for inverse problem solving, leveraging the recently introduced score-based generative models. Specifically, we first train a score-based generative model on medical images to capture their prior distribution. Given measurements and a physical model of the measurement process at test time, we introduce a sampling method to reconstruct an image consistent with both the prior and the observed measurements. Our method does not assume a fixed measurement process during training, and can thus be flexibly adapted to different measurement processes at test time. Empirically, we observe comparable or better performance to supervised learning techniques in several medical imaging tasks in CT and MRI, while demonstrating significantly better generalization to unknown measurement processes.

View Paper: https://www.semanticscholar.org/paper/49f6dbf4ead6a8a3d26f9cf218a654f2f3d1d896

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Denoising Diffusion Restoration Models (2022, background): “, degradation model is known during training) [42, 41, 13, 10, 54] and unsupervised settings [22, 26, 25, 21, 46, 47, 9].”
Score-based diffusion models for accelerated MRI (2021, methodology): “Song et al. (2022) only focused on solving simulated reconstructions from real-valued images, which limit the practicality.”
Unsupervised Medical Image Translation With Adversarial Diffusion Models (2022, methodology): “Given this potential, diffusion-based methods have recently been adopted for unimodal imaging tasks such as image reconstruction [45], [46], [47], [48], [49], unconditional image generation [50], and anomaly detection [51], [52].”
FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model (2023, background): “[27, 11, 14] attempt to use the cross-attention control to realize the conditional generation; [5, 6, 7, 8, 19, 24, 39, 41, 44, 43] directly modify the intermediate results to achieve zero-shot image restoration; [25] realizes image translation by adjusting the initial noisy images.”
Fast Sampling of Diffusion Models with Exponential Integrator (2022, background): “DMs have recently achieved impressive performances on a variety of tasks, including unconditional image generation [10, 30, 27, 6], text conditioned image generation [24, 26], text generation [12, 2], 3D point cloud generation [22], solving inverse problem [16, 32], etc.”
Diffusion models as plug-and-play priors (2022, background): “Both in [40] and [20], the authors modify the DDPM sampling algorithm, with knowledge of the linear degradation operator, to reconstruct an image consistent with the learned prior and given measurements.”
Diffusion models in medical imaging: A comprehensive survey (2022, methodology): “• SDEs achieve better results in multimodal conversion [49] • Cross-modality translation [50] • Conditional SDEs [50, 54, 58] • Solving linear inverse problems [52] • Using K-space data [147, 148] • Using manifold constraints [58] • Good initialization in the reverse process instead of random Gaussi”
Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction (2021, methodology): “Interestingly, diffusion models can go beyond unconditional image synthesis, and have been applied to conditional image generation, including super-resolution [5, 17, 25], inpainting [31, 34], MRI reconstruction [6, 13, 33], image translation [5, 19, 28], and so on.”
Improving Diffusion Models for Inverse Problems using Manifold Constraints (2022, methodology): “[34] Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon.”
Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, methodology): “An inverse problem aims to recover an unknown signal from observed measurements, and it is an important problem in medical image reconstruction of Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) [41, 42, 177, 227, 259].”
Diffusion Posterior Sampling for General Noisy Inverse Problems (2022, methodology): “…2022; Chung et al., 2022a), compressed-sensing MRI (CS-MRI) (Song et al., 2022; Chung & Ye, 2022; Chung et al., 2022b), computed tomography (CT) (Song et al., 2022; Chung et al., 2022a), etc., to our best knowledge, all works so far considered linear inverse problems only, and have not explored…”