NCBI disease corpus: A resource for disease name recognition and concept normalization

By Rezarta Islamaj Dogan, Robert Leaman, Zhiyong Lu. 2014

945 citations recorded by the source corpus.

Information encoded in natural language in biomedical literature publications is only useful if efficient and reliable ways of accessing and analyzing that information are available. Natural language processing and text mining tools are therefore essential for extracting valuable information, however, the development of powerful, highly effective tools to automatically detect central biomedical concepts such as diseases is conditional on the availability of annotated corpora. This paper presents the disease name and concept annotations of the NCBI disease corpus, a collection of 793 PubMed abstracts fully annotated at the mention and concept level to serve as a research resource for the biomedical natural language processing community. Each PubMed abstract was manually annotated by two annotators with disease mentions and their corresponding concepts in Medical Subject Headings (MeSH®) or Online Mendelian Inheritance in Man (OMIM®). Manual curation was performed using PubTator, which allowed the use of pre-annotations as a pre-step to manual annotations. Fourteen annotators were randomly paired and differing annotations were discussed for reaching a consensus in two annotation phases. In this setting, a high inter-annotator agreement was observed. Finally, all results were checked against annotations of the rest of the corpus to assure corpus-wide consistency. The public release of the NCBI disease corpus contains 6,892 disease mentions, which are mapped to 790 unique disease concepts. Of these, 88% link to a MeSH identifier, while the rest contain an OMIM identifier. We were able to link 91% of the mentions to a single disease concept, while the rest are described as a combination of concepts. In order to help researchers use the corpus to design and test disease identification methods, we have prepared the corpus as training, testing and development sets. To demonstrate its utility, we conducted a benchmarking experiment where we compared three different knowledge-based disease normalization methods with a best performance in F-measure of 63.7%. These results show that the NCBI disease corpus has the potential to significantly improve the state-of-the-art in disease name recognition and normalization research, by providing a high-quality gold standard thus enabling the development of machine-learning based approaches for such tasks.

Subfields: Genetics, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Adversarial Training for Large Neural Language Models (2020, background): “The NCBI disease corpus (Dogan et al., 2014) contains annotations of disease mentions from a collection of PubMed abstracts.”
TaggerOne: joint named entity recognition and normalization with semi-Markov Models (2016, methodology): “, 2015; Morgan, et al., 2008). The development of NER and normalization systems for diseases lagged behind genes and proteins for some time, primarily due to the lack of annotated corpora. Jimeno, et al. (2008) created a corpus of sentences that was expanded by Leaman, et al.”
Memory, Show the Way: Memory Based Few Shot Word Representation Learning (2018, methodology): “Domain Speciﬁc NER We use BioNLP11-species (Kim et al., 2011), AnatEMs (Pyysalo and Ananiadou, 2013) and NCBI-disease (Do˘gan et al., 2014) dataset, respectively from taxonomy, anatomy and pathology literatures.”
OAG-BERT: Pre-train Heterogeneous Entity-augmented Academic Language Models (2021, background): “Besides pre-training for general purposes, more and more language models are targeting at specific domains, such as BioBERT [26] for biomedical field and SciBERT [2] for academic field, which establish new state-of-the-art on many domain-related benchmarks such as named entity recognition [10, 32], ”
Self-Alignment Pretraining for Biomedical Entity Representations (2020, methodology): “B ERT models we use nearest neighbour search without further ﬁne-tuning on task data (unless model NCBI BC5CDR-d BC5CDR-c MedMentions AskAPatient COMETA @1 @5 @1 @5 @1 @5 @1 @5 @1 @5 @1 @5 vanilla B ERT (Devlin et al., 2019) scientiﬁc language social media language model NCBI BC5CDR-d BC5CDR-c MedMe”
A Survey of Deep Active Learning (2020, background): “[34, 109, 197, 199] CoNLL [184], NCBI [51], MedMentions [149], OntoNotes [167], DBLP, FZ, AG [147], Cora [228] Named entity recognition (NER) Question answering [13] CMDC [43], JabberWacky’s chatlogs9 Dialogue generation [104] Visual Genome [117], VQA [11] Visual question answer (VQA)”
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders (2021, neutral): “For biomedical entity linking, we use NCBI-disease (NCBI) (Doğan et al., 2014), BC5CDR-disease (BC5-d), BC5CDRchemical (BC5-c) (Li et al.”