MSˆ2: Multi-Document Summarization of Medical Studies

By Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang. 2021

147 citations recorded by the source corpus.

To assess the effectiveness of any medical intervention, researchers must conduct a time-intensive and manual literature review. NLP systems can help to automate or assist in parts of this expensive process. In support of this goal, we release MSˆ2 (Multi-Document Summarization of Medical Studies), a dataset of over 470k documents and 20K summaries derived from the scientific literature. This dataset facilitates the development of systems that can assess and aggregate contradictory evidence across multiple studies, and is the first large-scale, publicly available multi-document summarization dataset in the biomedical domain. We experiment with a summarization system based on BART, with promising early results, though significant work remains to achieve higher summarization quality. We formulate our summarization inputs and targets in both free text and structured forms and modify a recently proposed metric to assess the quality of our system’s generated summaries. Data and models are available at https://github.com/allenai/ms2.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “For example, DeYoung et al. (2021) investigate the problem of summarizing medical studies and their error analysis focuses on whether the effect of the study intervention was generated correctly (i.”
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, unknown): “3) Training Data: as mentioned earlier, the transition from PLMs to LLMs brings a significant shift from a model-centered approach to a data-centered approach.”
A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025, unknown): “Multiple benchmarks have been proposed to evaluate model sum-marisationcapabilitiesinthebiomedicaldomain,includingoverbiomedical literature 37 – 40 , medical forum conversations 41 , and radiology reports 22,42 .”