Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases

By Andrew Janowczyk, Anant Madabhushi. Journal of Pathology Informatics, 2016

1,296 citations recorded by the source corpus.

Background: Deep learning (DL) is a representation learning approach ideally suited for image analysis challenges in digital pathology (DP). The variety of image analysis tasks in the context of DP includes detection and counting (e.g., mitotic events), segmentation (e.g., nuclei), and tissue classification (e.g., cancerous vs. non-cancerous). Unfortunately, issues with slide preparation, variations in staining and scanning across sites, and vendor platforms, as well as biological variance, such as the presentation of different grades of disease, make these image analysis tasks particularly challenging. Traditional approaches, wherein domain-specific cues are manually identified and developed into task-specific "handcrafted" features, can require extensive tuning to accommodate these variances. However, DL takes a more domain agnostic approach combining both feature discovery and implementation to maximally discriminate between the classes of interest. While DL approaches have performed well in a few DP related image analysis tasks, such as detection and tissue classification, the currently available open source tools and tutorials do not provide guidance on challenges such as (a) selecting appropriate magnification, (b) managing errors in annotations in the training (or learning) dataset, and (c) identifying a suitable training set containing information rich exemplars. These foundational concepts, which are needed to successfully translate the DL paradigm to DP tasks, are non-trivial for (i) DL experts with minimal digital histology experience, and (ii) DP and image processing experts with minimal DL experience, to derive on their own, thus meriting a dedicated tutorial. Aims: This paper investigates these concepts through seven unique DP tasks as use cases to elucidate techniques needed to produce comparable, and in many cases, superior to results from the state-of-the-art hand-crafted feature-based classification approaches. Results : Specifically, in this tutorial on DL for DP image analysis, we show how an open source framework (Caffe), with a singular network architecture, can be used to address: (a) nuclei segmentation (F-score of 0.83 across 12,000 nuclei), (b) epithelium segmentation (F-score of 0.84 across 1735 regions), (c) tubule segmentation (F-score of 0.83 from 795 tubules), (d) lymphocyte detection (F-score of 0.90 across 3064 lymphocytes), (e) mitosis detection (F-score of 0.53 across 550 mitotic events), (f) invasive ductal carcinoma detection (F-score of 0.7648 on 50 k testing patches), and (g) lymphoma classification (classification accuracy of 0.97 across 374 images). Conclusion: This paper represents the largest comprehensive study of DL approaches in DP to date, with over 1200 DP images used during evaluation. The supplemental online material that accompanies this paper consists of step-by-step instructions for the usage of the supplied source code, trained models, and input data.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d847ee63fe234f9cc2a8be851ed511b7d1a8da36

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Lung and Colon Cancer Histopathological Image Dataset (LC25000) (2019, background): “[5] Machine Learning requires large image datasets for training.”
P3SGD: Patient Privacy Preserving SGD for Regularizing Deep CNNs in Pathological Image Classification (2019, background): “tissue classification [30, 10], lesion detection [17, 22], nuclei segmentation [38, 50, 26], etc.”
HistoQC: An Open-Source Quality Control Tool for Digital Pathology Slides (2019, background): “9,10 This is especially important for increasingly popular deep learning – and machine learning – based approaches, 11-16 which rely on well-annotated and relatively artefact-free images to learn underlying disease-speci ﬁ c representations.”
Detecting Cancer Metastases on Gigapixel Pathology Images (2017, methodology): “8 [12]), and verified visually that lymph node tissue was not discarded.”
Membership Inference Attacks on Machine Learning: A Survey (2021, neutral): “Classification & Generation IDC [81] 277,524 50×50×3 2 [215, 217] EyePACS [89] 88,702 N.”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, methodology): “Subsequently, the work from Janowczyk and Madabhushi [215] demonstrates the utility of DL approaches for identifying and segmenting a number of different histologic primitives, including lymphocytes, tubules, mitotic figures, cancer extent, and also for classifying different disease categories perta”
A survey on deep learning in medical image analysis (2017, methodology): “Global and local context are typically needed to perform accurate segmentation, such that multistream networks with different scales or non-uniformly sampled patches are used as in for example Kamnitsas et al. (2017) and Ghafoorian et al.”
Deep Learning for Classification of Colorectal Polyps on Whole-slide Images (2017, background): “released a survey of the applications of deep learning in pathology, exploring domains such as lymphocyte detection, mitosis detection, invasive ductal carcinoma detection, and lymphoma classification [17].”
A systematic study of the class imbalance problem in convolutional neural networks (2017, methodology): “One of the most commonly used method in deep learning [16, 30, 31, 32].”
Pathology Image Analysis Using Segmentation Deep Learning Algorithms. (2019, background): “Although there have previously been in-depth reviews on using machine learning methods, including deep learning, in digital pathology image analysis, 4,23 this is the ﬁ rst review of the implementations of segmentation deep learning algorithms for WSI analysis.”
AI in Medical Imaging Informatics: Current Challenges and Future Directions (2020, neutral): “A comprehensive review of these initiatives appears in [160]–[162].”
A review on deep learning in medical image analysis (2021, background): “When there is an abundance of labelled input data for one task that can be shared with another task with much less labelled data, multi-task learning can be a useful approach to problem-solving [74, 75].”
Computer-Aided Diagnosis in the Era of Deep Learning (2020, methodology): “Similar CAD tasks are also applicable to optical coherence tomography image analysis for eye diseases 38 or histopathological image analysis at the cellular level 39 .”