An Artificial Agent for Anatomical Landmark Detection in Medical Images

By Florin C. Ghesu, Bogdan Georgescu, Tommaso Mansi, Dominik Neumann, Joachim Hornegger, Dorin Comaniciu. 2016

149 citations recorded by the source corpus.

Fast and robust detection of anatomical structures or pathologies represents a fundamental task in medical image analysis. Most of the current solutions are however suboptimal and unconstrained by learning an appearance model and exhaustively scanning the space of parameters to detect a specific anatomical structure. In addition, typical feature computation or estimation of meta-parameters related to the appearance model or the search strategy, is based on local criteria or predefined approximation schemes. We propose a new learning method following a fundamentally different paradigm by simultaneously modeling both the object appearance and the parameter search strategy as a unified behavioral task for an artificial agent. The method combines the advantages of behavior learning achieved through reinforcement learning with effective hierarchical feature extraction achieved through deep learning. We show that given only a sequence of annotated images, the agent can automatically and strategically learn optimal paths that converge to the sought anatomical landmark location as opposed to exhaustively scanning the entire solution space. The method significantly outperforms state-of-the-art machine learning and deep learning approaches both in terms of accuracy and speed on 2D magnetic resonance images, 2D ultrasound and 3D CT images, achieving average detection errors of 1-2 pixels, while also recognizing the absence of an object from the image.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A survey on deep learning in medical image analysis (2017, methodology): “…set of 780 thousand images Cheng et al. (2016a) Various detection tasks US, CT AE, CNN Detection of breast lesions in US and pulmonary nodules in CT Ghesu et al. (2016a) Landmark detection US, CT, MRI CNN, DQN Reinforcement learning with CNN features, cardiac MR/US, head&neck CT Liu et al. (2016b)…”
Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification (2018, background): “Reinforcement learning has gained remarkable success recently in domains like playing Go [12], Atari games [13], and anatomical landmark detection in medical images [14], etc.”
Deep Reinforcement Learning for Active Breast Lesion Detection from DCE-MRI (2017, methodology): “In addition, this way of representing the bounding box means that we are able to process varying-size input bounding box, which is an advantage compared to [11].”
Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images (2018, methodology): “In our work, we focus on improving the learning of the navigating agent, keeping the basic formulation similar to the existing works [4]–[6].”
Deep learning in medical imaging and radiation therapy. (2018, background): “Ghesu et al. 139 formulated the 3D heart detection task as a regression problem, targeting at the 3D bounding box coordinates and affine transform parameters in transesophageal echocardiogram images.”
Robust Non-rigid Registration Through Agent-Based Action Learning (2017, methodology): “Ghesu et al. [2] used deep reinforcement learning (DRL) for landmark detection in 2-”
Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans (2019, methodology): “ing, which we reformulated for the context of anatomical landmark detection in previous work [21].”
Deep Learning in Medical Ultrasound Analysis: A Review (2019, background): “Note that reinforcement learning has also been combined with deep learning for anatomical (cardiac US) landmark detection [123].”