Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010

By Berry de Bruijn, Colin Cherry, Svetlana Kiritchenko, Joel D. Martin, Xiao-Dan Zhu. 2011

247 citations recorded by the source corpus.

Objective As clinical text mining continues to mature, its potential as an enabling technology for innovations in patient care and clinical research is becoming a reality. A critical part of that process is rigid benchmark testing of natural language processing methods on realistic clinical narrative. In this paper, the authors describe the design and performance of three state-of-the-art text-mining applications from the National Research Council of Canada on evaluations within the 2010 i2b2 challenge. Design The three systems perform three key steps in clinical information extraction: (1) extraction of medical problems, tests, and treatments, from discharge summaries and progress notes; (2) classification of assertions made on the medical problems; (3) classification of relations between medical concepts. Machine learning systems performed these tasks using large-dimensional bags of features, as derived from both the text itself and from external sources: UMLS, cTAKES, and Medline. Measurements Performance was measured per subtask, using micro-averaged F-scores, as calculated by comparing system annotations with ground-truth annotations on a test set. Results The systems ranked high among all submitted systems in the competition, with the following F-scores: concept extraction 0.8523 (ranked first); assertion detection 0.9362 (ranked first); relationship detection 0.7313 (ranked second). Conclusion For all tasks, we found that the introduction of a wide range of features was crucial to success. Importantly, our choice of machine learning algorithms allowed us to be versatile in our feature design, and to introduce a large number of features without overfitting and without encountering computing-resource bottlenecks.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuan Luo
2. Ozlem Uzuner
3. Peter Szolovits
4. Hua Xu
5. Noémie Elhadad
6. Shaodian Zhang
7. Min Jiang
8. Yonghui Wu

In Molecular Biology

1. Yuan Luo
2. Ozlem Uzuner
3. Peter Szolovits
4. Hua Xu
5. Noémie Elhadad
6. Shaodian Zhang
7. Min Jiang
8. Yonghui Wu

Citations in the corpus

• Recurrent Neural Networks for Classifying Relations in Clinical Notes (2017, neutral): “[15], we down sampled the training set to a PIP/ None ratio of 1:4.”
• Bridging semantics and syntax with graph algorithms —state-of-the-art of extracting biomedical relations (2015, unknown): “deBruijn et al.[127] applied a maximum entropy classifier with down sampling applied to balance the relation distribution.”
• Bridging semantics and syntax with graph algorithms - state-of-the-art of extracting biomedical relations (2017, methodology): “[94] applied a maximum entropy classifier with down sampling applied to balance the relation distribution.”
• Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network (2015, background): “Therefore, many participating teams including all the top five systems used ML-based methods [12-14].”
• Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts (2013, methodology): “…[25] was initially used by [30] for word sense disambiguation; on the other hand, the idea of classifying entities based on their context signatures [21] is also similar with distributional methods in word sense disambiguation [31], in which contexts of mentions are used to determine word senses.”