BioWordVec, improving biomedical word embeddings with subword information and MeSH

By Yijia Zhang, Qingyu Chen, Zhihao Yang, Hongfei Lin, Zhiyong Lu. 2019

497 citations recorded by the source corpus.

Distributed word representations have become an essential foundation for biomedical natural language processing (BioNLP), text mining and information retrieval. Word embeddings are traditionally computed at the word level from a large corpus of unlabeled text, ignoring the information present in the internal structure of words or any information available in domain specific structured resources such as ontologies. However, such information holds potentials for greatly improving the quality of the word representation, as suggested in some recent studies in the general domain. Here we present BioWordVec: an open set of biomedical word vectors/embeddings that combines subword information from unlabeled biomedical text with a widely-used biomedical controlled vocabulary called Medical Subject Headings (MeSH). We assess both the validity and utility of our generated word embeddings over multiple NLP tasks in the biomedical domain. Our benchmarking results demonstrate that our word embeddings can result in significantly improved performance over the previous state of the art in those challenging tasks. Design Type(s) data transformation objective • data integration objective • text processing and analysis objective Measurement Type(s) word representation Technology Type(s) Text_Mining Factor Type(s) Sample Characteristic(s) Design Type(s) data transformation objective • data integration objective • text processing and analysis objective Measurement Type(s) word representation Technology Type(s) Text_Mining Factor Type(s) Sample Characteristic(s) Machine-accessible metadata file describing the reported data (ISA-Tab format)

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets (2019, background): “One established trend is a form of word embed-dings that represent the semantic, using high dimensional vectors (Chiu et al., 2016; Wang et al., 2018c; Zhang et al., 2019).”