CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT

By Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y. Ng, Matthew P. Lungren. Conference on Empirical Methods in Natural Language Processing, 2020

547 citations recorded by the source corpus.

The extraction of labels from radiology text reports enables large-scale training of medical imaging models. Existing approaches to report labeling typically rely either on sophisticated feature engineering based on medical domain knowledge or manual annotations by experts. In this work, we introduce a BERT-based approach to medical image report labeling that exploits both the scale of available rule-based systems and the quality of expert annotations. We demonstrate superior performance of a biomedically pretrained BERT model first trained on annotations of a rule-based labeler and then finetuned on a small set of expert annotations augmented with automated backtranslation. We find that our final model, CheXbert, is able to outperform the previous best rules-based labeler with statistical significance, setting a new SOTA for report labeling on one of the largest datasets of chest x-rays.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/62f1a84ed518bc11a285a5d7dd78f66dbe06b5fc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

VinDr-CXR: An open dataset of chest X-rays with radiologist’s annotations (2020, background): “These labels were divided into 2 categories: local labels (1-22) and global labels (23-28).”
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing (2023, methodology): “Biomedical vision–language processing Paired medical image–report datasets were originally used for supervised learning via (typically) automated label extraction from clinical reports [33, 63, 70].”
AI in health and medicine (2022, methodology): “Such models have been used to improve performance on tasks such as learning from biomedical literature which drugs are known to interact with each other 55 or automatically labeling radiology reports 56 .”
Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review (2023, background): “In addition, a group of researchers [59] developed CheXbert, which employs BERT to classify free text radiological records.”
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation (2020, methodology): “We further examined whether these metrics can be used to estimate the performance of a clinical metric to see whether the proposed metrics can be used in an evaluation where a strong clinical information extraction system like CheXbert is not available.”
Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing (2022, methodology): “To link each proposal region with candidate phrases, we sample sentences from the report of each study by extracting the highest matching sentences to the annotated labels using scores of the CheXbert sentence classifier [68], and also use transcriptions of dictations when available [36].”
Shifting machine learning for healthcare from development to deployment and from models to data (2022, methodology): “BioBERT and ClinicalBERT have been harnessed to extract labels from radiology text reports, enabling an automatic clinical summarization tool and labeller 140 .”
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, methodology): “Hu et al. [80] utilized an anatomy-enhanced multimodal model to achieve state-of-the-art results in terms of the ROUGE and CheXbert [82] metrics.”