Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks

By Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas J Hawkins, Junzhou Huang. Medical Image Anal., 2020

551 citations recorded by the source corpus.

Traditional image-based survival prediction models rely on discriminative patch labeling which make those methods not scalable to extend to large datasets. Recent studies have shown Multiple Instance Learning (MIL) framework is useful for histopathological images when no annotations are available in classification task. Different to the current image-based survival models that limit to key patches or clusters derived from Whole Slide Images (WSIs), we propose Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling to efficiently learn imaging features from the WSI and then aggregate WSI-level information to patient-level. Attention-based aggregation is more flexible and adaptive than aggregation techniques in recent survival models. We evaluated our methods on two large cancer whole slide images datasets and our results suggest that the proposed approach is more effective and suitable for large datasets and has better interpretability in locating important patterns and features that contribute to accurate cancer survival predictions. The proposed framework can also be used to assess individual patient's risk and thus assisting in delivering personalized medicine.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f0c24d4ba514081a7bc77905e3e12998937ba68b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Faisal Mahmood
2. Ming Y. Lu
3. Drew F. K. Williamson
4. Tiffany Y. Chen
5. Richard J. Chen
6. Rahul G. Krishnan
7. Wei-Hung Weng
8. Chengkuan Chen

In Computer Vision and Pattern Recognition

1. Faisal Mahmood
2. Richard J. Chen
3. Rahul G. Krishnan
4. Ming Y. Lu
5. Wei-Hung Weng
6. Chengkuan Chen
7. Yicong Li
8. Tiffany Y. Chen

Citations in the corpus

• Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images (2021, methodology): “Recent work has demonstrated remarkable progress in using multiple Instance Learning (MIL) and other set-based deep learning approaches for learning tasks in gigapixel images [23, 5, 52, 51, 62, 69, 37, 36].”
• Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning (2022, background): “Recent variations of MIL have also evolved to extend the aggregation layers and scoring functions [17, 64, 68, 75, 77, 78, 85].”