Measuring classifier performance: a coherent alternative to the area under the ROC curve

By David J. Hand. Machine-mediated learning, 2009

1,071 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Health Information Management, Information Systems

View Paper: https://www.semanticscholar.org/paper/e9372b16ce7c3b11af9b6d1f5ab3b9319ab67d74

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Information Management

In Information Systems

Citations in the corpus

Machine Learning Risk Assessments in Criminal Justice Settings (2018, background): “Third, obtaining sufficiently accurate estimates of the requisite probabilities is a challenge, especially for machine learning applications (Mease et al. 2007; Lobo et al. 2008; Hand 2009).”
Deep learning for cardiovascular medicine: a practical primer. (2019, background): “Problems are that the c-index is only a measure of discrimination, ignores calibration indices and has no single universally accepted c-statistic cut-off or range of acceptable cstatistics.(92,93) This and other limitations can be addressed by calibration: dividing underlying continuous variables re”
Cost-sensitive learning methods for imbalanced data (2010, background): “This choice leads to the Kolmogorov-Smirnov statistic being the performance metric [36].”
TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (2018, background): “, ROC curves are misleading in an imbalanced dataset [11, 18]).”
Profit Driven Decision Trees for Churn Prediction (2017, background): “However, it has been shown by Hand (2009) that the AUC implicitly assumes that misclassiﬁcation errors carry the same costs which also alter across classiﬁers.”
Predicting mortgage default using convolutional neural networks (2018, methodology): “The H-measure, on the other hand, may accommodate expert knowledge regarding misclassification costs, whenever that is available.”
One-Pass AUC Optimization (2013, neutral): “learning literature (Provost et al., 1998; Provost and Fawcett, 2001; Elkan, 2001; Cortes and Mohri, 2004; Huang and Ling, 2005; Clemenćon et al., 2008; Hand, 2009; Flach et al., 2011).”
Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research (2015, methodology): “Furthermore, no previous study uses the H -measure, although it overcomes conceptual shortcomings of the AUC (Hand, 2009).”
A Comparison of MCC and CEN Error Measures in Multi-Class Prediction (2010, background): “or by adopting a performance measure as an indicator to derive similarities and differences, in particular as a function of the number of classes, class imbalance, and behaviour on randomized labels”
A Survey of Deep Learning Techniques for Weed Detection from Images (2021, methodology): “Different metrics are used to evaluate various characteristics of a classifier (Hand, 2009).”
An up-to-date comparison of state-of-the-art classification algorithms (2017, neutral): “…statistic, Cohen’s κ ( Carletta, 1996 ), Lift ( Giudici, 2005 ), Q-statistic ( Yule, 1900 ), Brier score ( Hernández-Orallo, Flach, & Ferri, 2012 ), H-measure ( Hand, 2009 ), Average/Relative information score ( Kononenko & Bratko, 1991 ), Partial Gini index ( Baesens et al., 2003 ), etc.”
Adaptive Machine Learning for Credit Card Fraud Detection (2015, background): “Hand [66] considers standard calculation of the AUC as inappropriate, since this translates into making an average of different misclassification costs for the two classes.”
Credit Card Fraud Detection: A Realistic Modeling and a Novel Learning Strategy (2018, methodology): “AUC can be estimated by means of the Mann–Whitney statistic [48] and its value can be interpreted as the probability that a classifier ranks frauds higher than genuine transactions [37].”