A deep learning framework for unsupervised affine and deformable image registration

By Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Hessam Sokooti, Marius Staring, Ivana Isgum. 2018

778 citations recorded by the source corpus.

HighlightsUnsupervised Deep Learning Image Registration (DLIR) is feasible for affine and deformable image registration.The method is unsupervised; no registration examples are necessary to train a ConvNet for image registration.Once a ConvNet is trained, image registration can be performed on unseen images in one‐shot.Registration, including image resampling, is near real‐time.Unsupervised DLIR yields image registration performance similar to a conventional approach. Graphical abstract Figure. No caption available. ABSTRACT Image registration, the process of aligning two or more images, is the core technique of many (semi‐)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for unsupervised affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity‐based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse‐to‐fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

TransMorph: Transformer for unsupervised medical image registration (2021, positive): “Note that the term ‘‘unsupervised’’ refers to the absence of ground-truth deformation fields, but the network still needs training (this is also known as ‘‘self-supervised’’). de Vos et al. (2019), Balakrishnan et al. (2018, 2019) are representative of unsupervised DNN-based methods.”
Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces (2019, background): “Building on the successful demonstration of these methods, several recent papers [9, 10, 23, 22, 44] explore unsupervised, or end-to-end, strategies.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, background): “More recently, a deep learning framework has been introduced in de Vos et al. (2019) for unsupervised affine and deformable image registration.”
CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration (2020, background): “[28] proposed affine and non-rigid image registration framework, and Lei et al.”
Recursive Cascaded Networks for Unsupervised Medical Image Registration (2019, methodology): “DLIR trains each cascade one by one, i.e., after ﬁxing the weights of previous cascades.”
Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks (2020, background): “To address this challenge, a preliminary study [25] proposes to stack multiple CNNs for direct affine and deformable image registrations, which are optimized separately.”
Deep learning in medical image registration: a review (2019, positive): “proposed an unsupervised affine and DIR framework by stacking multiple CNN into a larger network [23].”