Overview of BioCreative II gene normalization

By Alexander A. Morgan, Zhiyong Lu, Xinglong Wang, Aaron M. Cohen, Juliane Fluck, Patrick Ruch, A. Divoli, Katrin Fundel, Robert Leaman, Jörg Hakenberg and 10 more. Genome Biology, 2008

370 citations recorded by the source corpus.

Background:The goal of the gene normalization task is to link genes or gene products mentioned in the literature to biological databases. This is a key step in an accurate search of the biological literature. It is a challenging task, even for the human expert; genes are often described rather than referred to by gene symbol and, confusingly, one gene name may refer to different genes (often from different organisms). For BioCreative II, the task was to list the Entrez Gene identifiers for human genes or gene products mentioned in PubMed/MEDLINE abstracts. We selected abstracts associated with articles previously curated for human genes. We provided 281 expert-annotated abstracts containing 684 gene identifiers for training, and a blind test set of 262 documents containing 785 identifiers, with a gold standard created by expert annotators. Inter-annotator agreement was measured at over 90%.Results:Twenty groups submitted one to three runs each, for a total of 54 runs. Three systems achieved F-measures (balanced precision and recall) between 0.80 and 0.81. Combining the system outputs using simple voting schemes and classifiers obtained improved results; the best composite system achieved an F-measure of 0.92 with 10-fold cross-validation. A 'maximum recall' system based on the pooled responses of all participants gave a recall of 0.97 (with precision 0.23), identifying 763 out of 785 identifiers.Conclusion:Major advances for the BioCreative II gene normalization task include broader participation (20 versus 8 teams) and a pooled system performance comparable to human experts, at over 90% agreement. These results show promise as tools to link the literature with biological databases.

Subfields: Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/0da8a073174224b22486dde73bd09f31eb613cf7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

PubTator central: automated concept annotation for biomedical full text articles (2019, background): “The taggers for the remaining concept types were evaluated on corpora derived from abstracts (genes (45), diseases (46) and chemicals (41)).”
Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts (2013, background): “Recent supervised systems could efﬁciently ﬁnd gene names and clinical problems from certain type of texts with above 0.8 F score [6,12,13,14].”
PubMed and beyond: a survey of web tools for searching biomedical literature (2011, background): “Problems in these areas have received the most attention in the text mining community (68,69).”
Overview of BioCreative II gene mention recognition (2008, background): “This is the goal of the gene normalization task [2].”
NCBI disease corpus: A resource for disease name recognition and concept normalization (2014, background): “In addition, recent work in gene normalization has demonstrated that machine learning techniques provide a performance advantage if the necessary training data is available [14–16].”
DNorm: disease name normalization with pairwise learning to rank (2013, background): “The problem of concept normalization has seen substantial work for genes and proteins, as a result of a series of tasks that were part of the BioCreative competitions (Hirschman et al. , 2005a; Lu et al. , 2011; Morgan et al. , 2008).”
Evaluating Entity Linking with Wikipedia (2013, methodology): “The 2008 BioCreative workshop ran an entity linking challenge for biomedical text, which they termed Gene Normalisation ( gn ; Hirschman et al., 2005; Morgan et al., 2008).”