Adaptive Stochastic Gradient Descent Optimisation for Image Registration

By Stefan Klein, Josien P. W. Pluim, Marius Staring, Max A. Viergever. International Journal of Computer Vision, 2009

457 citations recorded by the source corpus.

We present a stochastic gradient descent optimisation method for image registration with adaptive step size prediction. The method is based on the theoretical work by Plakhov and Cruz (J. Math. Sci. 120(1):964–973, 2004). Our main methodological contribution is the derivation of an image-driven mechanism to select proper values for the most important free parameters of the method. The selection mechanism employs general characteristics of the cost functions that commonly occur in intensity-based image registration. Also, the theoretical convergence conditions of the optimisation method are taken into account. The proposed adaptive stochastic gradient descent (ASGD) method is compared to a standard, non-adaptive Robbins-Monro (RM) algorithm. Both ASGD and RM employ a stochastic subsampling technique to accelerate the optimisation process. Registration experiments were performed on 3D CT and MR data of the head, lungs, and prostate, using various similarity measures and transformation models. The results indicate that ASGD is robust to these variations in the registration framework and is less sensitive to the settings of the user-defined parameters than RM. The main disadvantage of RM is the need for a predetermined step size function. The ASGD method provides a solution for that issue.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/eae5297c99cc242654808b261b3ece3d5bed1ae2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Machine Learning for Wireless Networks with Artificial Intelligence: A Tutorial on Neural Networks (2017, methodology): “To reduce the time used for training when the size of the labeled data is very large, a stochastic gradient descent (SGD) algorithm [77] can be employed to update the weight values and bias.”
Fast parallel image registration on CPU and GPU for diagnostic classification of Alzheimer's disease (2013, methodology): “Considerable estimation time is still required by the ASGD optimizer (Klein et al., 2009b), which we addressed in separate work (Qiao et al., 2014).”
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial (2017, methodology): “ta is very large, then using backpropagation may be time consuming. To reduce the time used for training when the size of the labeled data is very large, a stochastic gradient descent (SGD) algorithm [77] can be employed to update the weight values and bias. The stochastic gradient descent algorithm”
Deformable Medical Image Registration: A Survey (2013, background): “Klein et al. [414] presented an adaptive step mechanism, while Bhagalia et al. [415] proposed an edge-driven importance sampling to improve the gradient approximation.”
elastix: A Toolbox for Intensity-Based Medical Image Registration (2010, methodology): “The latter suggestion is confirmed by performing a full rigid registration using the adaptive stochastic gradient descent optimization routine [55] on six patients from the RIRE database.”
Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video (2012, background): “There are many ways to pick the stepsize; [2] shows both diminishing and constant step-sizes, and we have also tried an adaptive step-size based on [14].”