An Unsupervised Learning Model for Deformable Medical Image Registration

By Guha Balakrishnan, Amy Zhao, Mert Rory Sabuncu, John V. Guttag, Adrian V. Dalca. 2018

703 citations recorded by the source corpus.

We present a fast learning-based algorithm for deformable, pairwise 3D medical image registration. Current registration methods optimize an objective function independently for each pair of images, which can be time-consuming for large data. We define registration as a parametric function, and optimize its parameters given a set of images from a collection of interest. Given a new pair of scans, we can quickly compute a registration field by directly evaluating the function using the learned parameters. We model this function using a CNN, and use a spatial transform layer to reconstruct one image from another while imposing smoothness constraints on the registration field. The proposed method does not require supervised information such as ground truth registration fields or anatomical landmarks. We demonstrate registration accuracy comparable to state-of-the-art 3D image registration, while operating orders of magnitude faster in practice. Our method promises to significantly speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is available at https://github.com/balakg/voxelmorph.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, background): “[246]38; and a deep learning framework for unsupervised affine and deformable image registration [247].”
Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration (2022, methodology): “We plug the CPSTN into FlowNet and VoxelMorph as their improved versions, then adopt them to conduct cross-modality IR-VIS image registration.”
An unsupervised convolutional neural network-based algorithm for deformable image registration (2018, methodology): “However, the MRI image sets used in these studies have similar image acquisition environments, and thus allow for shallower network design and less point matching discrimination than would be required in mixed modality image registration (Hu et al 2017, Balakrishnan et al 2018).”
Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation (2019, methodology): “This mature field spans optimization-based methods [4, 7, 67, 70], and recent learning-based methods [8, 9, 20, 42, 62, 72, 80].”
Deep learning in medical image registration: a survey (2019, methodology): “[7, 8] proposed a general framework for unsupervised image registration, which can be either unimodal or multimodal theoretically.”
Deep learning in medical image registration: a review (2019, methodology): “[181] Brain 3D-3D No MR Deformable [5, 6, 21] Brain 3D-3D No MR Deformable”
Recursive Cascaded Networks for Unsupervised Medical Image Registration (2019, methodology): “We quantify the performance by the Dice score [22] based on the segmentation of some anatomical structure, between the warped moving image and the ﬁxed image, as done in [8, 19].”
CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration (2020, methodology): “…Methods: In order to verify the improved performance of the proposed method, we employed several comparative methods that show the state-of-the-art performance in the image registration: Elastix [8], SyN [7] by Advanced Normalization Tools (ANTs) [36], VoxelMorph [16], and MS-DIRNet [18].”
Diffusion models in medical imaging: A comprehensive survey (2022, methodology): “The learned latent code along the source image is fed into the deformation module, adopted from [143], and creates deformation fields.”
TransMorph: Transformer for unsupervised medical image registration (2021, positive): “The detailed parameter settings used for each method were as follows: • VoxelMorph 8 (Balakrishnan et al., 2018, 2019): We employed two variants of VoxelMorph , the second variant doubles the number of convolution filters in the first variant; they are designated as VoxelMorph -1 and -2,…”
VoxelMorph: A Learning Framework for Deformable Medical Image Registration (2018, neutral): “puter Vision and Pattern Recognition [6].”
3D Deep Learning on Medical Images: A Review (2020, methodology): “Compared to the traditional optimization approaches (ANTs [107], Elastix [112] and VoxelMorph-2 [113]), their method was 880 times faster, with state-of-the-art performance.”
Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration (2018, methodology): “A recent paper avoids these pitfalls, but still does not provide topology-preserving guarantees or probabilistic uncertainty estimates, which yield meaningful information for downstream image analysis [5].”
Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks (2020, methodology): “Baseline Methods We compare our method with two conventional approaches (denoted as Demons [27] and SyN [2]) and two cutting edge DLIR methods (denoted as VM[4] and DIF-VM [5]).”
A Survey of Deep Learning for Scientific Discovery (2020, methodology): “For example, [12] develop an unsupervised method to perform alignment for Brain MRIs.”
Recent advances and clinical applications of deep learning in medical image analysis (2021, background): “…ROI segmentation (Alom et al., 2018; Yu et al., 2019; Fan et al., 2020), medical object detection (Rijthoven et al., 2018; Mei et al., 2021; Nair et al., 2020; Zheng et al., 2015), and cross-modality image registration (Simonovsky et al., 2016; Sokooti et al., 2017; Balakrishnan et al., 2018).”
Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces (2019, methodology): “We implement our method as part of the VoxelMorph package [9], available online at http://voxelmorph.”