Artificial Intelligence for Multimodal Data Integration in Oncology

By J. Lipkova, Richard J. Chen, Bowen Chen, Ming Y. Lu, Matteo Barbieri, Daniel Shao, Anurag Vaidya, Chengkuan Chen, Luoting Zhuang, Drew F. K. Williamson and 3 more. Cancer Cell, 2022

486 citations recorded by the source corpus.

In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, genomics to electronic-health records, each one providing additional insights. Current AI models, however, operate mainly in the realm of single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. At the same time, AI models are capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges towards clinical adoption and discuss possible emerging solutions.

Subfields: Artificial Intelligence, Health Informatics, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/dc28434a6d10ae9b151dce1fed9591ddfcaf2b81

Citations in the corpus

• Artificial intelligence for digital and computational pathology (2023, unknown): “To overcome this limitation, one option is to perform 3D reconstruction from serially sectioned and aligned H&E tissue sections to visualize the entire tissue at subcellular resolution, without the need for special sample preparation protocols”
• Multimodal data fusion for cancer biomarker discovery with deep learning (2023, unknown): “However, multimodal approaches are starting to emerge with encouraging solutions not only for interpretability but also for discovery of associations between modalities 147,150 .”