A visual–language foundation model for pathology image analysis using medical Twitter

By Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J. Montine, James Y. Zou, James Y. Zou. Nature Medicine, 2023

826 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Molecular Biology, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/0f0024bfef037b97b324b97150ee022c178d6282

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Empowering Biomedical Discovery with AI Agents (2024, positive): “Instruction tuning [59] can be used for the former by training the LLM to follow human instruction through prompt examples, including dialogues that incorporate biological reasoning [60].”
A foundation model for clinical-grade computational pathology and rare cancers detection (2024, neutral): “…large datasets is even greater for applications with inadequate quantities of data to develop bespoke models, as is the case for the detection of uncommon or rare tumor types, as well as for less common diagnostic tasks such as the prediction of specific genomic alterations, clinical outcomes…”
A whole-slide foundation model for digital pathology from real-world data (2024, neutral): “Prior work on pathology vision–language modelling tends to focus on tile-level alignment of pathology images and text, as their studies were limited by the sources of image–text pairs (textbook examples 7 or Twitter data 8 ).”
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs (2023, methodology): “As a result, biomedical vision-language foundation models [3–6] have been developed to collectively model biomedical images (e.g., digital pathology and radiology images) and biomedical text (e.g., scientific papers and clinical notes).”
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, neutral): “6M image-caption pairs Homepage CheXpert [299] Multimodel 224,316 chest radiographs with associated reports Homepage PadChest [300] Multimodel 160,000 images with related text Homepage MIMIC-CXR Multimodel 227,835 imaging studies for 64,588 patients Homepage PMC-15M [239] Multimodel 15 million Figur”
CNS-Obsidian: A Neurosurgical Vision-Language Model Built From Scientific Publications. (2025, unknown): “Since high-quality medical information is scarce in public datasets 12 and uncurated internet sources such as Twitter 26 , scientific publications offer an immediate solution, particularly for multimodal paired image-text context, which is rare outside of curated sources.”