Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets
By Nathan Greenberg, Trapit Bansal, Pat Verga, Andrew McCallum. 2018
67 citations recorded by the source corpus.
Extracting typed entity mentions from text is a fundamental component to language understanding and reasoning. While there exist substantial labeled text datasets for multiple subsets of biomedical entity types—such as genes and proteins, or chemicals and diseases—it is rare to find large labeled datasets containing labels for all desired entity types together. This paper presents a method for training a single CRF extractor from multiple datasets with disjoint or partially overlapping sets of entity types. Our approach employs marginal likelihood training to insist on labels that are present in the data, while filling in “missing labels”. This allows us to leverage all the available data within a single model. In experimental results on the Biocreative V CDR (chemicals/diseases), Biocreative VI ChemProt (chemicals/proteins) and MedMentions (19 entity types) datasets, we show that joint training on multiple datasets improves NER F1 over training in isolation, and our methods achieve state-of-the-art results.
Subfields: Artificial Intelligence, Molecular Biology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jonas Pfeiffer
- Iryna Gurevych
- Edwin Simpson
- Xuanjing Huang
- Ruotian Ma
- Natawut Monaikul
- Minlong Peng
- Simone Filice
In Molecular Biology
Citations in the corpus
- Continual Learning for Named Entity Recognition (2021, methodology): “Greenberg et al. (2018) present a method for training a single CRF extractor from multiple datasets.”
- Low Resource Multi-Task Sequence Tagging - Revisiting Dynamic Conditional Random Fields (2020, methodology): “Greenberg et al. (2018) train a single CRF from multiple datasets using marginal likelihood training to mitigate missing labels.”
- Simplify the Usage of Lexicon in Chinese NER (2019, background): “The bidirectional long-short term memory network (BiLSTM) is one of the most commonly used architectures for sequence modeling (Ma and Hovy, 2016; Lample et al., 2016; Greenberg et al., 2018).”