Data-efficient and weakly supervised computational pathology on whole-slide images

By Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, Faisal Mahmood. Nature Biomedical Engineering, 2020

2,191 citations recorded by the source corpus.

Deep-learning methods for computational pathology require either manual annotation of gigapixel whole-slide images (WSIs) or large datasets of WSIs with slide-level labels and typically suffer from poor domain adaptation and interpretability. Here we report an interpretable weakly supervised deep-learning method for data-efficient WSI processing and learning that only requires slide-level labels. The method, which we named clustering-constrained-attention multiple-instance learning (CLAM), uses attention-based learning to identify subregions of high diagnostic value to accurately classify whole slides and instance-level clustering over the identified representative regions to constrain and refine the feature space. By applying CLAM to the subtyping of renal cell carcinoma and non-small-cell lung cancer as well as the detection of lymph node metastasis, we show that it can be used to localize well-known morphological features on WSIs without the need for spatial labels, that it overperforms standard weakly supervised classification algorithms and that it is adaptable to independent test cohorts, smartphone microscopy and varying tissue content. A data-efficient and interpretable deep-learning method for the multi-class classification of whole-slide images that relies only on slide-level labels is applied to the detection of lymph node metastasis and to cancer subtyping.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3e358c3033908a9506e7f1e3cf29283e359f43d6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Artificial intelligence for digital and computational pathology (2023, methodology): “sensitivity from WSIs, allowing the identification of patients who do not require further testing 133 .”
Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images (2021, methodology): “Due to the enormous gigapixel resolutions of WSIs, many approaches adopt a two-stage multiple instance learning-based (MIL) approach for tractable representation learning of WSIs, in which: 1) instance-level feature representations are extracted from randomly sampled image patches in the WSI, and th”
Efficient Classification of Very Large Images with Tiny Objects (2021, methodology): “We consider the streaming CNN [38], CLAM [31], MIL [4] and MRMIL [29], which we previously briefly described.”
DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification (2022, methodology): “023 CLAM-MB [23] 0.878 0 .”
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology (2022, methodology): “In this case, the aggregation function is not a simple average but applies a weight to each tile 29,31 (Fig.”
TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication (2021, methodology): “There were also other pioneering works [10, 11, 12, 13, 14] in weakly supervised WSI diagnosis.”
Deep learning in cancer diagnosis, prognosis and treatment selection (2021, methodology): “Moreover, the model employed an explainability method called attention [109, 110] to assign diagnostic relevance scores to image regions and revealed that regions with cancer cells contributed most to both metastasis and origin decision making [58].”
Algorithm fairness in artificial intelligence for medicine and healthcare (2023, background): “…(in the form of attention weights) are given to semantic high-level visual concepts, such as the ‘blue crown’ and ‘blue back’ attributes that co-occur frequently in blue jay birds, or tumour-lymphocyte colocalization in histology slides as a biomarker of favourable cancer prognosis 346,347 .”
Deep learning in histopathology: the path to the clinic (2021, background): “…learning approach 92 , in which at least one small region in the image is considered to contain morphological information that is needed to classify the image; for example, the presence of a single small region containing cancer is sufficient to label the entire WSI as containing cancer 75,93,94 .”
Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning (2022, methodology): “We follow the study design in [53]; we examined the following tasks evaluated using a 10fold cross-validated AUC: 1) Invasive Ductal (IDC) versus Invasive Lobular Carcinoma (ILC) in Invasive Breast Carcinoma (BRCA) subtyping, 2) Lung Adenocarcinoma (LUAD) versus Lung Squamous Cell Carcinoma (LUSC) i”