The use of receiver operating characteristic curves in biomedical informatics

By Thomas A. Lasko, Jui G. Bhagwat, Kelly H. Zou, Lucila Ohno-Machado. Journal of Biomedical Informatics, 2005

884 citations recorded by the source corpus.

Receiver operating characteristic (ROC) curves are frequently used in biomedical informatics research to evaluate classification and prediction models for decision support, diagnosis, and prognosis. ROC analysis investigates the accuracy of a model's ability to separate positive from negative cases (such as predicting the presence or absence of disease), and the results are independent of the prevalence of positive cases in the study population. It is especially useful in evaluating predictive models or other tests that produce output values over a continuous range, since it captures the trade-off between sensitivity and specificity over that range. There are many ways to conduct an ROC analysis. The best approach depends on the experiment; an inappropriate approach can easily lead to incorrect conclusions. In this article, we review the basic concepts of ROC analysis, illustrate their use with sample calculations, make recommendations drawn from the literature, and list readily available software.

Subfields: Artificial Intelligence, Physiology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6cdc99a9beaca55e5913d97a6fe1c650a9d8be47

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Physiology

In Statistics and Probability

Citations in the corpus

Prediction of response to antiretroviral therapy by human experts and by the EuResist data‐driven expert system (the EVE study) (2010, neutral): “…between speciﬁcity and sensitivity for labelling a treatment as successful was evaluated by receiver operating characteristics (ROC) analysis [17], where the area under the ROC curve (AUC) was used as an indicator of the performance of a binary classiﬁer (success/failure), with AUC values up…”
The DIARETDB1 Diabetic Retinopathy Database and Evaluation Protocol (2007, background): “Receiving operating curve (ROC) is a natural selection due to its popularity and proven applicability in similar computer vision tasks, such as face recognition [7], object class recognition [3] and medical research [8].”
Computational Phenotype Discovery Using Unsupervised Feature Learning over Noisy, Sparse, and Irregular Clinical Data (2013, methodology): “…model on training and test sets using the area under the ROC curve, calculated with the nonparametric empirical method, with 95% confidence intervals calculated using 30 iterations of the bias-corrected and accelerated bootstrap [59], all using MATLAB stats toolbox implementations.”
Predicting plant invaders in the Mediterranean through a weed risk assessment system (2010, background): “The closer the area under the ROC curve is to one, the better the screening tool’s ability to differentiate between the two groups (Lasko et al. 2005).”
The Immune Epitope Database and Analysis Resource in Epitope Discovery and Synthetic Vaccine Design (2017, background): “The area under this ROC curve is the AUC value and that has a number of important statistical properties (9).”
DIARETDB 1 diabetic retinopathy database and evaluation protocol (2007, background): “Receiving operating curve (ROC) is a natural selection due to its popularity and proven applicability in similar computer vision tasks, such as face recognition [7], object class recognition [3] and medical research [9].”
What's under the ROC? An Introduction to Receiver Operating Characteristics Curves (2007, methodology): “Even if the AUCs aren’t the same, but the lines cross, it’s necessary to choose between them by using a finer-grained test, called the partial AUC, or pAUC.(9,13) Rather than calculating the AUC over the entire range of the test, we focus our attention on that portion of curves that is of most inter”
Consistent accuracy of the Australian weed risk assessment system across varied geographies (2008, methodology): “The closer the area under the curve is to 0.5 (a diagonal line), the worse the tool’s ability to distinguish between the two groups (i.e. frequency distributions of the two groups tend to overlap) (Lasko et al ., 2005).”