Large language models are few-shot clinical information extractors
By Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, David A. Sontag. Conference on Empirical Methods in Natural Language Processing, 2022
516 citations recorded by the source corpus.
A long-running goal of the clinical NLP community is the extraction of important variables trapped in clinical notes. However, roadblocks have included dataset shift from the general domain and a lack of public clinical corpora and annotations. In this work, we show that large language models, such as InstructGPT (Ouyang et al., 2022), perform well at zero- and few-shot information extraction from clinical text despite not being trained specifically for the clinical domain. Whereas text classification and generation performance have already been studied extensively in such models, here we additionally demonstrate how to leverage them to tackle a diverse set of NLP tasks which require more structured outputs, including span identification, token-level sequence classification, and relation extraction. Further, due to the dearth of available data to evaluate these systems, we introduce new datasets for benchmarking few-shot clinical information extraction based on a manual re-annotation of the CASI dataset (Moon et al., 2014) for new tasks. On the clinical extraction tasks we studied, the GPT-3 systems significantly outperform existing zero- and few-shot baselines.
Subfields: Artificial Intelligence, Molecular Biology
View Paper: https://www.semanticscholar.org/paper/686d9ee744fa013cc21cdd86acd864c936e9e456
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Molecular Biology
Citations in the corpus
- Large language models for generative information extraction: a survey (2023, background): “…information from some specific domains, such as mulitmodal [57, 94, 166, 167], multilingual [83, 133, 163], medical [85, 91, 162, 163, 171, 172, 179, 183, 184, 185, 186, 187, 188, 234, 235], scientific [47, 80, 180, 181, 182], astronomical [164, 173], historical [126, 189], and legal [78, 89, 115].”
- Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, methodology): “For example, Agrawal et al. [95] used GPT-3 for NER task on the CASI dataset and showed that GPT-3 was able to outperform the baseline model by observing a single input-output pair.”
- The TRIPOD-LLM reporting guideline for studies using large language models (2025, unknown): “This adaptability is commonly achieved through supervised fine-tuning or few-shot learning methods, which allow LLMs to handle new tasks with minimal examples 15,16 .”
- The shaky foundations of large language models and foundation models for electronic health records (2023, background): “For example, a CLaM could extract drug names from a doctor ’ s note 28 , automatically reply to patient questions 29 , summarize medical dialogues 30 , or predict mechanical ventilation needs based on clinical notes 31 .”
- A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, unknown): “LLMs were trained on unlabeled data, with most of their knowledge derived from a vast amount of textual information.”