Image Similarity and Tissue Overlaps as Surrogates for Image Registration Accuracy: Widely Used but Unreliable

By Torsten Rohlfing. IEEE Transactions on Medical Imaging, 2011

418 citations recorded by the source corpus.

The accuracy of nonrigid image registrations is commonly approximated using surrogate measures such as tissue label overlap scores, image similarity, image difference, or transformation inverse consistency error. This paper provides experimental evidence that these measures, even when used in combination, cannot distinguish accurate from inaccurate registrations. To this end, we introduce a “registration” algorithm that generates highly inaccurate image transformations, yet performs extremely well in terms of the surrogate measures. Of the tested criteria, only overlap scores of localized anatomical regions reliably distinguish reasonable from inaccurate registrations, whereas image similarity and tissue overlap do not. We conclude that tissue overlap and image similarity, whether used alone or together, do not provide valid evidence for accurate registrations and should thus not be reported or accepted as such.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/9e503d524a8eaa0d60e698ee0f974a6dac8fbe4a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A survey of medical image registration - under review (2016, positive): “We would, however, like to draw the readers’ attention to the study by Rohlﬁng (2012) , which shows that the approach of evaluating registration algorithms on the basis of image similarity and tissue overlap measures has severe shortcomings and hence should be used with caution.”
An unsupervised convolutional neural network-based algorithm for deformable image registration (2018, background): “Features that correspond well with respect to intensity, however, do not necessarily correspond well in regards to anatomy (Rohlfing 2012, Wu et al 2012a).”
Deep MRI brain extraction: A 3D convolutional neural network for skull stripping (2016, methodology): “0 came from the Internet Brain Segmentation Repository7 (IBSR) and consisted of N = 18 nT1w scans (0.94 × 1.5 × 0.94 mm) from healthy subjects (Rohlfing, 2012).”
The Lov´asz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (methodology): “Data and Model In order to test the Lov ´ asz-Softmax loss on a different type of images, we consider the publicly available dataset provided by the Internet Brain Segmentation Repository (IBSR) [24].”
Deformable Medical Image Registration: A Survey (2013, background): “…for image registration ( i.e. , Non-rigid Image Regis-tration Evaluation Project - NIREP [450]) and the increasing understanding regarding the use of surrogate measures for the measurement of the accuracy of registration [451] will further facilitate the comparison between different algorithms.”
Scalable High Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning (2015, background): “patches that show similar, or even the same, distribution of intensity values do not guarantee the two points are corresponded from an anatomical point of view [4], [12]–[14].”