A shared task involving multi-label classification of clinical free text

By J. Pestian, Chris Brew, P. Matykiewicz, D. Hovermale, N. Johnson, K. Cohen, Wlodzislaw Duch. 2007

425 citations recorded by the source corpus.

This paper reports on a shared task involving the assignment of ICD-9-CM codes to radiology reports. Two features distinguished this task from previous shared tasks in the biomedical domain. One is that it resulted in the first freely distributable corpus of fully anonymized clinical text. This resource is permanently available and will (we hope) facilitate future research. The other key feature of the task is that it required categorization with respect to a large and commercially significant set of labels. The number of participants was larger than in any previous biomedical challenge task. We describe the data production process and the evaluation measures, and give a preliminary analysis of the results. Many systems performed at levels approaching the inter-coder agreement, suggesting that human-like performance on this task is within the reach of currently available technologies.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Online Metric Learning for Multi-Label Classification (2020, methodology): “To evaluate the performance of our proposed online metric learning algorithm, we conduct experiments on four benchmark datasets: Corel5k, Enron, Medical and Emotions.”
Semi-supervised clinical text classification with Laplacian SVMs: an application to cancer case management (2013, positive): “The number of labeled instances from our dataset is roughly equivalent in size to that of clinical text classification challenges [26,27].”
A Survey of Deep Active Learning (2020, background): “Text classification tasks also face the challenge of excessive labeling costs, such as patent classification [60, 125] and clinical text classification [64, 73, 160].”
Revisiting Transformer-based Models for Long Document Classification (2022, background): “The task of assigning most relevant ICD codes to a document, e.g., radiology report (Pestian et al., 2007), death certiﬁcate (Koop-man et al., 2015) or discharge summary (Johnson et al., 2016), as a whole, has a long history of development (Farkas and Szarvas, 2008).”
On label dependence and loss minimization in multi-label classification (2012, methodology): “In the case of the SCENE and MEDICAL datasets, it seems that both risk minimizers coincide.”
Automatic Diagnosis Coding of Radiology Reports: A Comparison of Deep Learning and Conventional Classification Methods (2017, methodology): “ICD9 dataset is an open challenge dataset published by the Computational Medicine Center in 2007 ( Pestian et al., 2007).”
Machine Learning and Decision Support in Critical Care (2016, background): “ogy reports and gave participants the task of assigning two codes from a set of 45 ICD-9 codes [35].”