Harnessing multimodal data integration to advance precision oncology

By K. Boehm, Pegah Khosravi, R. Vanguri, Jianjiong Gao, Sohrab P. Shah. Nature Reviews. Cancer, 2021

563 citations recorded by the source corpus.

Advances in quantitative biomarker development have accelerated new forms of data-driven insights for patients with cancer. However, most approaches are limited to a single mode of data, leaving integrated approaches across modalities relatively underdeveloped. Multimodal integration of advanced molecular diagnostics, radiological and histological imaging, and codified clinical data presents opportunities to advance precision oncology beyond genomics and standard molecular techniques. However, most medical datasets are still too sparse to be useful for the training of modern machine learning techniques, and significant challenges remain before this is remedied. Combined efforts of data engineering, computational methods for analysis of heterogeneous data and instantiation of synergistic data models in biomedical research are required for success. In this Perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods. Advancing along this direction will result in a reimagined class of multimodal biomarkers to propel the field of precision oncology in the coming decade. This Perspective proposes that data from multiple modalities, including molecular diagnostics, radiological and histological imaging and codified clinical data, should be integrated by multimodal machine learning models to advance the prognosis and treatment management of patients with cancer.

Subfields: Artificial Intelligence, Cancer Research, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/a0022ee86e99981f0cde48fb4258b8ae21b96c6e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics (2023, unknown): “The distribution of diseases (i.e., the number of relevant cases) is as follows: COPD (4,912), bronchiectasis (676), pneumothorax (2,538), pneumonia (21,409), ILD (3,283), tuberculosis (938), lung cancer (2,651) and pleural effusion (4,713).”
Artificial intelligence for digital and computational pathology (2023, background): “The increasing availability of spatially resolved sequencing technologies such as spatial transcriptomics has ignited interest in understanding the connections between specific morphologies and single-cell molecular profiles.”
Artificial intelligence in drug development (2025, unknown): “Robust validation and responsible data curation will facilitate biomarker identification and application, supporting future drug development and disease treatment.”
Multimodal biomedical AI (2022, background): “…of cancer, overcoming challenges related to data access, sharing and accurate labeling could potentially lead to impactful tools that leverage the combination of personalized omics data with histopathology, imaging and clinical data to inform clinical trajectories and improve patient outcomes 23 .”
From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment. (2023, methodology): “Regularization is a technique that controls the size of the parameters of a model to prevent overfitting and encourage sparse feature usage.(2) Weak supervision involves using data with limited or noisy label information.”
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology (2022, background): “Currently, data integration from multiple sources, including histopathology, radiology and genomics, is widely regarded as a prerequisite for the use of AI to improve patient outcomes in precision oncology 69,70 .”