Deep learning with word embeddings improves biomedical named entity recognition

By Maryam Habibi, Leon Weber, Mariana Neves, David Luis Wiegandt, Ulf Leser. Bioinform., 2017

549 citations recorded by the source corpus.

Motivation: Text mining has become an important tool for biomedical research. The most fundamental text‐mining task is the recognition of biomedical named entities (NER), such as genes, chemicals and diseases. Current NER methods rely on pre‐defined features which try to capture the specific surface properties of entity types, properties of the typical local context, background knowledge, and linguistic information. State‐of‐the‐art tools are entity‐specific, as dictionaries and empirically optimal feature sets differ between entity types, which makes their development costly. Furthermore, features are often optimized for a specific gold standard corpus, which makes extrapolation of quality measures difficult. Results: We show that a completely generic method based on deep learning and statistical word embeddings [called long short‐term memory network‐conditional random field (LSTM‐CRF)] outperforms state‐of‐the‐art entity‐specific NER tools, and often by a large margin. To this end, we compared the performance of LSTM‐CRF on 33 data sets covering five different entity classes with that of best‐of‐class NER tools and an entity‐agnostic CRF implementation. On average, F1‐score of LSTM‐CRF is 5% above that of the baselines, mostly due to a sharp increase in recall. Availability and implementation: The source code for LSTM‐CRF is available at https://github.com/glample/tagger and the links to the corpora are available at https://corposaurus.github.io/corpora/. Contact: habibima@informatik.hu‐berlin.de

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/91d4498849fe1966a629cddb187d3cd62eedb2ca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (2018, background): “Studies have shown the importance of such pre-trained word embeddings for neural network based NER systems (Habibi et al., 2017), and similarly for pre-trained character embeddings in character-based languages like Chinese (Li et al., 2015; Yin et al., 2016).”
Enhancing Clinical Concept Extraction with Contextual Embedding (2019, background): “Up to now, the most prominent model for clinical concept extraction is a bidirectional Long Short-Term Memory with Conditional Random Field (Bi-LSTM CRF) architecture (Habibi et al., 2017; Florez et al., 2018; Chalapathy et al., 2016).”
From POS tagging to dependency parsing for biomedical event extraction (2018, result): “ssociated with gold tags NN, JJ and NNS; many are abbreviations which are also out-of-vocabulary. It is typically dicult for character-level word embeddings to capture those unseen abbreviated words [40]. On both GENIA and CRAFT, BiLSTM-CRF with character-level word embeddings obtains the highest ac”
BioBERT: a pre-trained biomedical language representation model for biomedical text mining (2019, background): “Note: The number of annotations from Habibi et al. (2017) and Zhu et al.”
CollaboNet: collaboration of deep neural networks for biomedical named entity recognition (2018, methodology): “mbeddings capture semantic and syntactic meanings of words, they have been widely used in various natural language processing tasks including named entity recognition. The experiment of Habibi et al. [18] showedthat wordembeddings trained on biomedical corpora notably improved the performance of Bio”
A Survey of Deep Learning for Scientific Discovery (2020, background): “It has already found interesting uses in aiding with the automatic analysis of scientiﬁc texts, such as drug name recognition systems [131], biomedical named entity recognition [73], identifying important concepts in materials science [230] and even detecting chemical-protein interactions [37].”
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning (2020, methodology): “, 2011), using CNN to learn word representations and CRF to decode labels; (2) LSTMCRF (Habibi et al., 2017), using a Bi-LSTM to learn word representations; (3) GRAM-CNN (Zhu et al.”
FedNER: Medical Named Entity Recognition with Federated Learning (2020, methodology): “…NER methods, including: (1) CNN-CRF (Col-lobert et al., 2011), using CNN to learn word representations and CRF to decode labels; (2) LSTM-CRF (Habibi et al., 2017), using a Bi-LSTM to learn word representations; (3) GRAM-CNN (Zhu et al., 2017), using CNN to learn both character and word…”