The CHEMDNER corpus of chemicals and drugs and its annotation principles

By Martin Krallinger, O. Rabal, F. Leitner, M. Vázquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M. Lowe and 43 more. Journal of Cheminformatics, 2015

444 citations recorded by the source corpus.

The automatic extraction of chemical information from text requires the recognition of chemical entity mentions as one of its key steps. When developing supervised named entity recognition (NER) systems, the availability of a large, manually annotated text corpus is desirable. Furthermore, large corpora permit the robust evaluation and comparison of different approaches that detect chemicals in documents. We present the CHEMDNER corpus, a collection of 10,000 PubMed abstracts that contain a total of 84,355 chemical entity mentions labeled manually by expert chemistry literature curators, following annotation guidelines specifically defined for this task. The abstracts of the CHEMDNER corpus were selected to be representative for all major chemical disciplines. Each of the chemical entity mentions was manually labeled according to its structure-associated chemical entity mention (SACEM) class: abbreviation, family, formula, identifier, multiple, systematic and trivial. The difficulty and consistency of tagging chemicals in text was measured using an agreement study between annotators, obtaining a percentage agreement of 91. For a subset of the CHEMDNER corpus (the test set of 3,000 abstracts) we provide not only the Gold Standard manual annotations, but also mentions automatically detected by the 26 teams that participated in the BioCreative IV CHEMDNER chemical mention recognition task. In addition, we release the CHEMDNER silver standard corpus of automatically extracted mentions from 17,000 randomly selected PubMed abstracts. A version of the CHEMDNER corpus in the BioC format has been generated as well. We propose a standard for required minimum information about entity annotations for the construction of domain specific corpora on chemical and drug entities. The CHEMDNER corpus and annotation guidelines are available at: http://www.biocreative.org/resources/biocreative-iv/chemdner-corpus/

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/932117813af889104cafb457ebbc52a38d8b64eb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Molecular Biology

Citations in the corpus

tmChem: a high performance approach for chemical named entity recognition and normalization (2015, methodology): “The full CHEMDNER corpus and the complete annotation guidelines are available for down-load (after free registration) [20].”
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (2018, methodology): “(Krallinger et al., 2015) introduced the similar CHEMDNER task 7 focusing more on chemical and drug entities like trivial, systematic, abbreviation, formula, family, identiﬁer, etc. Biology and microbiology NER datasets 8 (Hirschman et al., 2005; Bossy et al., 2013; Del˙eger et al., 2016) have been…”
BioMedBERT: A Pre-trained Biomedical Language Model for QA and IR (2020, unknown): “15411 BC4CHEMD (Krallinger et al., 2015) Drug/Chem.”
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020, methodology): “For example, the CHEMPROT dataset for extracting relations between chemicals and proteins focuses on abstracts of recently-published, high-impact articles from hand-selected PubMed categories (Krallinger et al., 2017, 2015).”
PubTator central: automated concept annotation for biomedical full text articles (2019, methodology): “We improved the performance of TaggerOne for chemicals by re-training the model using a combination of the BioCreative V CDR corpus (41) and the CHEMDNER corpus (42).”
BioBERT: a pre-trained biomedical language representation model for biomedical text mining (2019, methodology): “Note that although there are several other recently introduced high quality biomedical NER datasets (Mohan and Li, 2019), we use datasets that are frequently used by many biomedical NLP researchers, which makes it much easier to compare our work with theirs. The RE datasets contain gene–disease rela”
Bio-Megatron: Larger Biomedical Domain Language Model (2020, methodology): “Relation Extraction The ChemProt (Krallinger et al., 2015) dataset contains sentences from PubMed abstracts, where chemical-protein interaction types are annotated as ﬁve categories.”
BioCreative V CDR task corpus: a resource for chemical disease relation extraction (2016, background): “Despite the existence of many biomedical corpora (see (9) for a brief review) including a few specifically targeting diseases (10–12) and chemicals (13), there were none that fulfilled the following content criteria: (i) inclusion of instances of chemical-dis-ease relation annotations that are…”
An extensive review of tools for manual annotation of documents (2019, methodology): “[4]) or used a custom (private) tool or an adapted (non-available) version of an existing one [12, 13].”
CollaboNet: collaboration of deep neural networks for biomedical named entity recognition (2018, methodology): “Datasets We used 5 datasets (BC2GM [33], BC4CHEMD [34], BC5CDR [35–38], JNLPBA [22], NCBI [21]), all of which were collected by Crichton et al [24] (Table 1).”