Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks

By Yipeng Hu, Eli Gibson, Tom Kamiel Magda Vercauteren, Hashim Uddin Ahmed, Mark Emberton, Caroline M. Moore, J. Alison Noble, Dean C. Barratt. 2017

16 citations recorded by the source corpus.

In this paper, we describe how a patient-specific, ultrasound-probe-induced prostate motion model can be directly generated from a single preoperative MR image. Our motion model allows for sampling from the conditional distribution of dense displacement fields, is encoded by a generative neural network conditioned on a medical image, and accepts random noise as additional input. The generative network is trained by a minimax optimisation with a second discriminative neural network, tasked to distinguish generated samples from training motion data. In this work, we propose that 1) jointly optimising a third conditioning neural network that pre-processes the input image, can effectively extract patient-specific features for conditioning; and 2) combining multiple generative models trained separately with heuristically pre-disjointed training data sets can adequately mitigate the problem of mode collapse. Trained with diagnostic T2-weighted MR images from 143 real patients and 73,216 3D dense displacement fields from finite element simulations of intraoperative prostate motion due to transrectal ultrasound probe pressure, the proposed models produced physically-plausible patient-specific motion of prostate glands. The ability to capture biomechanically simulated motion was evaluated using two errors representing generalisability and specificity of the model. The median values, calculated from a 10-fold cross-validation, were 2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced approach demonstrates the feasibility of applying state-of-the-art machine learning algorithms to generate organ motion models from patient images, and shows significant promise for future research.

Subfields: Computer Vision and Pattern Recognition, Radiation, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Visual Feature Attribution Using Wasserstein GANs (2017, methodology): “Although some of the above models use 3D data, the examined volumes are usually relatively small [23], or the networks operate in a patch-wise fashion [40].”
Generative Adversarial Network in Medical Imaging: A Review (2018, methodology): “Hu et al. (2017a) used combined WGAN and InfoGAN for unsupervised cell-level feature representation learning in histopathology images whereas Yi et al. (2018) combined WGAN and CatGAN for unsupervised and semi-supervised feature representation learning for dermoscopy images.”
Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks (2018, methodology): “GANs have been used to generate a motion model from a single preoperative MRI [10], upsample a lowresolution fundus image [13], create a synthetic head CT from a brain MRI [16], and synthesizing T2-weight MRI from T1-weighted ones (and vice-versa) [5].”