ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

By Mark Neumann, Daniel King, Iz Beltagy, Bridger Waleed Ammar. 2019

870 citations recorded by the source corpus.

Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is a critically important application area of natural language processing, for which there are few robust, practical, publicly available models. This paper describes scispaCy, a new Python library and models for practical biomedical/scientific text processing, which heavily leverages the spaCy library. We detail the performance of two packages of models released in scispaCy and demonstrate their robustness on several tasks and datasets. Models and code are available at https://allenai.github.io/scispacy/.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Medical & Health AI

Citations in the corpus

SciBERT: A Pretrained Language Model for Scientific Text (2019, methodology): “We split sentences using ScispaCy (Neumann et al., 2019), 2 which is optimized for scientiﬁc text.”
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020, methodology): “Abstracts and body paragraphs from biomedical and computer science articles were used after sentence splitting using scispaCy (Neumann et al., 2019).”
S2ORC: The Semantic Scholar Open Research Corpus (2020, methodology): “We tokenize and sentencize the text using scispaCy (Neumann et al., 2019).”
Publicly Available Clinical BERT Embeddings (2019, methodology): “To process these notes, we split all notes into sections, then used Scispacy (Neumann et al., 2019)”
Large language models are few-shot clinical information extractors (2022, methodology): “To this end, we leveraged the Python library ScispaCy with the en_core_sci_sm package for entity recognition (Neumann et al., 2019, details in Appendix C.4).”
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies (2021, methodology): “We first identify a set of entities characterising the domain at hand using a domain-specific entity recogniser (SciSpacy (Neumann et al., 2019)), and then employ a subset of those entities to drive the masking strategy while fine-tuning (Figure 1).”
SciBERT: Pretrained Contextualized Embeddings for Scientific Text (2019, positive): “To split the documents into sentences, we use ScispaCy (Neumann et al., 2019),4 which is optimized for scientific text.”
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset (2020, methodology): “…following query generation technique proved to be effective: when constructing the keyword query for a given topic, we take the non-stopwords from the query ﬁeld and further expand them with terms belonging to named entities extracted from the question ﬁeld using ScispaCy (Neumann et al., 2019).”
Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021, methodology): “concept extraction systems include cTAKES, MetaMap, MedLEE, and scispaCy [2, 10, 32, 42].”
TLDR: Extreme Summarization of Scientific Documents (2020, methodology): “We use ScispaCy (Neumann et al., 2019) for sentence segmentation where necessary 6.”
Syntactic Search by Example (2020, methodology): “Over a pubmed corpus, annotated with the SciSpacy (Neumann et al., 2019) pipeline:”
CORD-19: The Covid-19 Open Research Dataset (2020, methodology): “Wang et al. (2020) releases a supplementary dataset over CORD-19 by augmenting text with entity mentions predicted from multiple techniques, including ScispaCy and weak supervision based on the NLM’s Uniﬁed Medical Language System (UMLS) Metathesaurus (Bodenreider, 2004).”
ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model (2023, background): “scales due to architecture design and hardware support, although some efficient algorithms are proposed for the medical domain.(20,33,34) One of the most important LLMs is T5, with 11 billion parameters proposed by Google.”
OAG-BERT: Pre-train Heterogeneous Entity-augmented Academic Language Models (2021, methodology): “We clean up and sentencize the corpus with SciSpacy [33].”