Image analysis and machine learning in digital pathology: Challenges and opportunities

By Anant Madabhushi, George Lee. Medical Image Anal., 2016

1,005 citations recorded by the source corpus.

With the rise in whole slide scanner technology, large numbers of tissue slides are being scanned and represented and archived digitally. While digital pathology has substantial implications for telepathology, second opinions, and education there are also huge research opportunities in image computing with this new source of “big data”. It is well known that there is fundamental prognostic data embedded in pathology images. The ability to mine “sub-visual” image features from digital pathology slide images, features that may not be visually discernible by a pathologist, offers the opportunity for better quantitative modeling of disease appearance and hence possibly improved prediction of disease aggressiveness and patient outcome. However the compelling opportunities in precision medicine offered by big digital pathology data come with their own set of computational challenges. Image analysis and computer assisted detection and diagnosis tools previously developed in the context of radiographic images are woefully inadequate to deal with the data density in high resolution digitized whole slide images. Additionally there has been recent substantial interest in combining and fusing radiologic imaging and proteomics and genomics based measurements with features extracted from digital pathology images for better prognostic prediction of disease aggressiveness and patient outcome. Again there is a paucity of powerful tools for combining disease specific features that manifest across multiple different length scales. The purpose of this review is to discuss developments in computational image analysis tools for predictive modeling of digital pathology images from a detection, segmentation, feature extraction, and tissue classification perspective. We discuss the emergence of new handcrafted feature approaches for improved predictive modeling of tissue appearance and also review the emergence of deep learning schemes for both object detection and tissue classification. We also briefly review some of the state of the art in fusion of radiology and pathology images and also combining digital pathology derived image measurements with molecular “omics” features for better predictive modeling. The review ends with a brief discussion of some of the technical and computational challenges to be overcome and reflects on future opportunities for the quantitation of histopathology.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/fc7127cda3ca2c87ccb6e4ec9f8b4ddf843132c9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Deep Learning for Classification of Colorectal Polyps on Whole-slide Images (2017, methodology): “In the past years, computational methods have been developed to assist pathologists in the analysis of microscopic images [12, 26, 29].”
Deep learning in histopathology: the path to the clinic (2021, background): “The history of CPATH before deep learning has been reviewed elsewhere 13 , as has the application of deep learning for histopathology from a technical perspective 14 .”
Deep learning with multimodal representation for pancancer prognosis prediction (2019, background): “Similarly, recent work has shown that quantitative analysis of histopathology images using computer vision algorithms can provide additional information on top of what can be discerned by pathologists (Madabhushi and Lee, 2016).”
Lung and Colon Cancer Histopathological Image Dataset (LC25000) (2019, methodology): “[3][4] ML allows representative images to be used to train a computer to recognize patterns from labeled photographs.”
Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology (2017, background): “Fusing traditional image sources with *omics data calls for new solutions (Madabhushi and Lee, 2016), particularly if we want to go towards personalized medicine (Holzinger, 2014c).”
A Review on a Deep Learning Perspective in Brain Cancer Classification (2019, background): “It is anticipated that the design of 3D WSI scanners may be available soon [120].”
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer (2017, neutral): “Training (n = 270 images) RUMC 54 16 100 35 35 2 (1-20) 170”
HistoQC: An Open-Source Quality Control Tool for Digital Pathology Slides (2019, methodology): “9,10 This is especially important for increasingly popular deep learning – and machine learning – based approaches, 11-16 which rely on well-annotated and relatively artefact-free images to learn underlying disease-speci ﬁ c representations.”
Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association (2019, methodology): “Supervised ML techniques, in which an algorithm is trained using ground truth labels, are particularly effective in image segmentation (detection of specific objects) and classification (such as tumor diagnosis) tasks [14].”
Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images (2018, background): “T are with the Department of Pathology at University Hospitals Coventry and Warwickshire, Coventry, UK efficient processing, analysis and management of the tissue specimens [2].”
Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning (2018, background): “org July 2018 | Volume 8 | Article 240 DCNNs can enhance extraction and recognition of CLE diagnostic features that may be integrated into the standard brain tumor classification protocols similarly to the current research flow in the whole-slide digital image analysis for personalized cancer care (”