A deep learning model to predict RNA-Seq expression of tumours from whole slide images

By B. Schmauch, A. Romagnoni, E. Pronier, C. Saillard, P. Maillé, J. Calderaro, A. Kamoun, M. Sefta, Sylvain Toldo, Mikhail Zaslavskiy and 4 more. Nature Communications, 2020

475 citations recorded by the source corpus.

Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3- and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability. RNA-sequencing of tumour tissue can provide important diagnostic and prognostic information but this is costly and not routinely performed in all clinical settings. Here, the authors show that whole slide histology slides—part of routine care—can be used to predict RNA-sequencing data and thus reduce the need for additional analyses.

Subfields: Artificial Intelligence, Biophysics, Cancer Research

View Paper: https://www.semanticscholar.org/paper/4021945d940b65494e385061fa52aa183e4f5c32

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Cancer Research

Citations in the corpus

Artificial intelligence in histopathology: enhancing cancer research and clinical oncology (2022, background): “Yet another work focused on gene expression and provided significant predictions for more than 10 4 genes for various cancers, with results varying greatly across different datasets 79 .”
Deep learning in histopathology: the path to the clinic (2021, background): “…currently impossible to capture by sole visual inspection of histopathology slides, such as prediction of response to chemotherapy or immunotherapy, or even future events such as recurrence or survival 97–104 as well as the presence of genetic mutations 95,105–110 or molecular subtypes 30,111,112 .”
Deep learning in cancer diagnosis, prognosis and treatment selection (2021, methodology): “Another DL model called HE2RNA used weakly supervised learning to infer gene expression from histopathology images, which were then used to infer MSI status in colorectal cancer [68].”
Artificial intelligence for digital and computational pathology (2023, background): “5) and translational considerations for clinical deployment (Box 3).”