The relationship between Precision-Recall and ROC curves

By Jesse Davis, Mark H. Goadrich. 2006

6,855 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Information Systems, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Information Systems

In Evaluation & Benchmarks

Citations in the corpus

Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions (2015, positive): “PR-AUC(AP): It has been pointed out in [2] that for the dataset where the number of positive and negative samples are imbalanced, the precision-recall curve has shown to be a better metric compared to ROC.”
The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, background): “The AUROC measures the probability that a detector will assign a random anomaly a higher score than a random non-anomalous policy (Davis & Goadrich, 2006).”
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (2016, methodology): “Faced with this issue, we employ the Area Under the Receiver Operating Characteristic curve (AU-ROC) metric, which is a threshold-independent performance evaluation (Davis & Goadrich, 2006).”
Early Methods for Detecting Adversarial Images (2016, methodology): “Faced with this issue, we employ the Area Under the Receiver Operating Characteristic curve (AU-ROC) metric, which is a threshold-independent performance evaluation (Davis & Goadrich, 2006).”
Opportunities and obstacles for deep learning in biology and medicine (2017, background): “Likewise, auROC evaluates the average recall across all false positive rate (FPR) thresholds, which is often a highly misleading metric in class-imbalanced domains [71,328].”
Introducing “best single template” models as reference baseline for the Continuous Automated Model Evaluation (CAMEO) (2019, background): “space, a large change in the false positives has a small effect, while it is clearly captured in the PR space performance.(28) Consequently CAMEO QE considers both analysis domains and offers a performance analysis based on partial ROC and partial PR.”
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks (2017, background): “(3) AUROC is the Area Under the Receiver Operating Characteristic curve, which is also a threshold-independent metric (Davis & Goadrich, 2006).”
Deep Anomaly Detection with Outlier Exposure (2018, background): “The AUROC can be thought of as the probability that an anomalous example is given a higher OOD score than a in-distribution example (Davis & Goadrich, 2006).”
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty (2019, methodology): “We evaluate each OOD detector by using the area under the receiver operating characteristic curve (AUROC) [Davis and Goadrich, 2006].”
A Review of Relational Machine Learning for Knowledge Graphs (2015, background): “For data with a large number of negative examples (as it is typically the case for knowledge graphs), it has been shown that AUC-PR can give a clearer picture of an algorithm’s performance than AUC-ROC [124].”
A systematic study of the class imbalance problem in convolutional neural networks (2017, background): “Even though it was proved that a curve dominates in ROC space if and only if it dominates in PR space [99], they are clearly different.”
Predicting In-Hospital Mortality of ICU Patients: The PhysioNet/Computing in Cardiology Challenge 2012 (2012, background): “The optimization of the PR curve area has been suggested as a good criterion for unevenly distributed classiﬁcation problems [8].”
Evaluation of CNN-based Automatic Music Tagging Models (2020, background): “However, the area under precision-recall curve (PR-AUC) can be more informative for evaluation on highly skewed datasets [13].”
OpenGAN: Open-Set Recognition via Open Data Generation (2021, positive): “However, when trained on pixels, CLS pix works poorly in AUROC due to overﬁtting to high-dimensional raw images, but performs decently in F1.”
Benchmarking Metagenomics Tools for Taxonomic Classification. (2019, background): “By ranging the abundance threshold from 0–1.0, the area under this precision-recall curve (AUPR) outputs a single metric to aggregate precision and recall scores (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015).”