The use of the area under the ROC curve in the evaluation of machine learning algorithms

By Andrew P. Bradley. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, background): “We recommend that people report the area under the receiver operating characteristic curve (ROC AUC) [6] as the main metric, but also report accuracy since the test set is balanced and that metric is easier to interpret.”
Strategies for Pre-training Graph Neural Networks (2019, methodology): “Speciﬁcally, in the chemistry datasets, we see from Table 1 that our Context Prediction + Graph-level multi-task supervised pre-training strategy gives the most promising performance, leading to an increase in average ROC-AUC of 7.2% over non-pre-trained baseline and 4.2% over graph-level multi-task”
Mining with rarity: a unifying framework (2004, methodology): “Perhaps the most common is ROC analysis and the associated use of the area under the ROC curve (AUC) to assess overall classification performance [4; 40].”
Machine learning predictive models for mineral prospectivity: an evaluation of neural networks, random forest, regression trees and support vector machines (2015, unknown): “5 is considered as random guessing (Bradley, 1997).”
Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences (2020, background): “The ROC curve is plotted such that TPR is on vertical axis and FPR is on the horizontal axis (the line TPR = FPR represents a random guess of a specific class) [105].”
Support Vector Data Description (2004, background): “The fraction of the outliers which is accepted (EII), is averaged over varying EI (Bradley, 1997):”
Using AUC and accuracy in evaluating learning algorithms (2005, methodology): “Bradley [3] has compared popular machine learning algorithms using AUC and found that AUC exhibits several desirable properties compared to accuracy.”
All-paths graph kernel for protein-protein interaction extraction with evaluation of cross-corpus learning (2008, background): “Due to this and other beneficial properties for comparative evaluation, the usage of AUC for performance evaluation has been recently advocated in the machine learning community (see e.g. [31]).”
Membership Inference Attacks against Generative Models with Probabilistic Fluctuation (unknown): “As shown in Table 2, we first summarize the AUC [Bradley, 1997] and ASR [Duan et al. , 2023] metrics for all base-lines and PFAMI against two generative models over two datasets.”
A Least-squares Approach to Direct Importance Estimation (2009, methodology): “Since there is a trade-off between the detection rate and the detection accuracy, we adopt the area under the ROC curve (AUC) as our error metric (Bradley, 1997).”