Overview of BioCreative II gene mention recognition
By Larry Smith, Lorraine K. Tanabe, Rie Ando, Cheng-Ju Kuo, I-Fang Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M. Friedrich, Kuzman Ganchev and 24 more. 2008
541 citations recorded by the source corpus.
Nineteen teams presented results for the Gene Mention Task at the BioCreative II Workshop. In this task participants designed systems to identify substrings in sentences corresponding to gene name mentions. A variety of different methods were used and the results varied with a highest achieved F1 score of 0.8721. Here we present brief descriptions of all the methods used and a statistical analysis of the results. We also demonstrate that, by combining the results from all submissions, an F score of 0.9066 is feasible, and furthermore that the best result makes use of the lowest scoring submissions.
Subfields: Artificial Intelligence, Molecular Biology, Information Extraction, Computational Biology & Bioinformatics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zhiyong Lu
- Robert Leaman
- Chih-Hsuan Wei
- Jinhyuk Lee
- Jaewoo Kang
- Leon Weber
- Aram Galstyan
- WonJin Yoon
In Molecular Biology
In Information Extraction
- Robert Leaman
- Zhiyong Lu
- Chih-Hsuan Wei
- Qingcai Chen
- Jaewoo Kang
- Karin M. Verspoor
- Alfonso Valencia
- Tapio Salakoski
In Computational Biology & Bioinformatics
- Robert Leaman
- Zhiyong Lu
- Chih-Hsuan Wei
- Tolga Can
- Jaewoo Kang
- Alfonso Valencia
- Sampo Pyysalo
- Xiaolong Wang
Citations in the corpus
- BioBERT: a pre-trained biomedical language representation model for biomedical text mining (2019, methodology): “We used the pre-processed versions of all the NER datasets provided by Wang et al. (2018) except the 2010 i2b2/VA, JNLPBA and Species800 datasets.”
- BioMedBERT: A Pre-trained Biomedical Language Model for QA and IR (2020, background): “79842 BC2GM (Smith et al., 2008) Gene/Protein 20703 JNLPBA (Kim et al., 2004) Gene/Protein 35460 Relation extraction is another sub-task of information extraction which involves detection and classifi-cation of relationships between named entity mentions in a biomedical corpus.”
- Multitask learning and benchmarking with clinical time series data (2017, methodology): “ence intervals of the score. Bootstrapping have been used to estimate the standard deviations of the evaluation measures [50], to compute statistically significant differences between different models [51] and to report 95% bootstrap confidence intervals for the models [25,52]. To estimate a 95% confid”
- TarBase 6.0: capturing the exponential growth of miRNA targets with experimental support (2011, methodology): “The subsequent named entity identification process was divided into two distinct steps: in the first step, genes were identified using AIIA Lab’s AIIAGMT, one of the highest ranking named entity recognition systems in BioCreative challenges (18).”
- Deep learning meets ontologies: experiments to anchor the cardiovascular disease ontology in the biomedical literature (2018, methodology): “For BioCreative II, Smith et al. [46] mention three tasks: gene mention (GM), gene normalisation (GN), and protein-protein interaction (PPI); the first two are within the scope of NER, whilst the third is a relation extraction task that has NER as a subtask [47].”
- The CHEMDNER corpus of chemicals and drugs and its annotation principles (2015, background): “The competitive performance of ensemble systems has been demonstrated for instance for the recognition of gene mentions [46] or the detection of protein interactions [47], showing in some cases that even low scoring runs can positively contribute to the ensemble system performance.”
- tmChem: a high performance approach for chemical named entity recognition and normalization (2015, background): “Many successful named entity recognition systems have improved performance by exploiting the complementary strengths of multiple models [5].”
- Distributional Semantics Resources for Biomedical Text Processing (2013, methodology): “…mention tagging experiments using three biomedical domain corpora representing various tagging tasks: the BioCreative II Gene Mention task corpus (Smith et al., 2008) (gene and protein names), the Anatomical Entity Mention (AnEM) corpus (Ohta et al., 2012) (anatomical entity mentions) and the…”
- Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, methodology): “…prompt engineering method to evaluate ChatGPT’s performance on biomedical NER and RE in the BLURB benchmark datasets including BC5CDR-chemical [94], BC5CDR-disease [94], NCBI-disease [99], BC2GM [100], JNLPBA [101], ChemProt [102], DDI [103], and GAD [104] in a zero-shot or few-shot manner.”
- Evaluating Word Representation Features in Biomedical Named Entity Recognition Tasks (2014, methodology): “For example, almost all top-ranked teams in BioCreAtIvE II GM and JNLPBA challenges utilized CRF [2, 3].”
- Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation (2021, methodology): “, 2005) for protein relation extraction (REL) • BC2GM - BioCreative II gene mention dataset (Smith et al., 2008) for NER task • ChEMU - Chemical Reactions from Patents”