End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network

By Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, Ivana Isgum. DLMIA/ML-CDS@MICCAI, 2017

485 citations recorded by the source corpus.

In this work we propose a deep learning network for deformable image registration (DIRNet). The DIRNet consists of a convolutional neural network (ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet analyzes a pair of fixed and moving images and outputs parameters for the spatial transformer, which generates the displacement vector field that enables the resampler to warp the moving image to the fixed image. The DIRNet is trained end-to-end by unsupervised optimization of a similarity metric between input image pairs. A trained DIRNet can be applied to perform registration on unseen image pairs in one pass, thus non-iteratively. Evaluation was performed with registration of images of handwritten digits (MNIST) and cardiac cine MR scans (Sunnybrook Cardiac Data). The results demonstrate that registration with DIRNet is as accurate as a conventional deformable image registration method with short execution times.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e9d754ef02442c550822a4a257e82e7425cb18ee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep learning in medical image registration: a survey (2019, methodology): “Further, de Vos et al. [26] used NCC to train an FCN to perform the deformable registration of 4D cardiac cine MR volumes.”
Recent advances and clinical applications of deep learning in medical image analysis (2021, result): “DLIR is another famous unsupervised registration framework (de Vos et al., 2019), which is an extension of the previous work (de Vos et al., 2017).”
VoxelMorph: A Learning Framework for Deformable Medical Image Registration (2018, background): “slices [40], and support only small transformations [40].”
Deformable Image Registration Using Cue-aware Deep Regression Network (2018, methodology): “Recently, unsupervised deep learning is also adopted for registration, in which the similarity metrics (e.g., normalized cross-correlation [53], [54]) are directly used to train the deep network in backpropagation.”
TransMorph: Transformer for unsupervised medical image registration (2021, unknown): “Recently, the focus has been shifted towards developing unsupervised methods that do not depend on ground-truth deformation fields (Balakrishnan et al., 2019; Dalca et al., 2019 et al., 2021; de Vos et al., 2019, 2017; Lei et al., 2020; Chen et al., 2020; Zhang, 2018).”
Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces (2019, background): “Building on the successful demonstration of these methods, several recent papers [9, 10, 23, 22, 44] explore unsupervised, or end-to-end, strategies.”
Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey (2020, unknown): “To address the shortcoming of the manual registration process, DL implementations in the image registration process have improved the productivity of the image registration process (de Vos, Berendsen, Viergever, Staring, & I ˇ sgum, 2017).”
An Unsupervised Learning Model for Deformable Medical Image Registration (2018, methodology): “Others [14] employ regularization only implicitly determined by interpolation methods.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Another method in de Vos et al. (2017) similarly trained a deep network to regress and output the parameters of spatial transformation, which can then generate the displacement field to warp the moving image to the target image.”
Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration (2018, background): “Some preliminary papers [9,18] explore unsupervised strategies that build on the spatial transformer network [15], but are only demonstrated with constrained deformation models such as affine or small displacement fields.”