Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation

By Ozan Oktay, Enzo Ferrante, Konstantinos Kamnitsas, Mattias Paul Heinrich, Wenjia Bai, Jose Caballero, S. Cook, A. de Marvao, T. Dawes, D. O’Regan and 3 more. IEEE Transactions on Medical Imaging, 2017

732 citations recorded by the source corpus.

Incorporation of prior knowledge about organ shape and location is key to improve performance of image analysis approaches. In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition. The highly constrained nature of anatomical objects can be well captured with learning-based techniques. However, in most recent and promising techniques such as CNN-based segmentation it is not obvious how to incorporate such prior knowledge. State-of-the-art methods operate as pixel-wise classifiers where the training objectives do not incorporate the structure and inter-dependencies of the output. To overcome this limitation, we propose a generic training strategy that incorporates anatomical prior knowledge into CNNs through a new regularisation model, which is trained end-to-end. The new framework encourages models to follow the global anatomical properties of the underlying anatomy (e.g. shape, label structure) via learnt non-linear representations of the shape. We show that the proposed approach can be easily adapted to different analysis tasks (e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models. The applicability of our approach is shown on multi-modal cardiac data sets and public benchmarks. In addition, we demonstrate how the learnt deep models of 3-D shapes can be interpreted and used as biomarkers for classification of cardiac pathologies.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/33f7b95903eeefdde37fabf9655b4de089d53bd2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation (2019, methodology): “For the task of myocardium segmentation, the authors report performance superior to 3D-ACNN Oktay et al. (2017),”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, background): “[168]) and were nicely included in an anatomicallyconstrained deep learning strategy by [169].”
Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation (2019, background): “rance features, implicit shape information and high-level context together for further improving the volumetric segmentation performance. Generally, 3D CNNs have superiority in medical image analysis [39], [40] against 2D CNNs, due to the 3D nature of many medical images. However, 3D CNNs contain a ”
Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey (2020, unknown): “The authors in Oktay et al. (2017) propose a novel technique to overcome this challenge by implementing a generic training strategy that embeds prior knowledge into the CNNs using a regularisation model which is trained thoroughly.”
Automatic Multi-organ Segmentation on Abdominal CT with Dense V-networks (2018, unknown): “Imposing shape [29] and topological [30] constraints between specified organs also remains”
Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography (2019, methodology): “[19] used CNNs to segment the 3D”
An application of cascaded 3D fully convolutional networks for medical image segmentation (2018, background): “Furthermore, additional anatomical constraints could be included in order to guarantee topologically correct segmentation results (BenTaieb and Hamarneh, 2016; Oktay et al., 2017).”
Medical Image Segmentation Using Deep Learning: A Survey (2020, methodology): “[124] proposed a novel and general method to combine a priori knowledge of shape and label structure into the anatomically constrained neural networks”