A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

By David J. Hand, Robert J. Till. 2001

2,393 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Health Information Management, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Information Management

In Information Systems

Citations in the corpus

Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing (2014, positive): “We measure performance both in terms of accuracy, which is the percentage of samples for which the algorithm correctly predicted genotype, and AUCROC, which is the multi-class area under the ROC curve defined by Hand and Till [17].”
Interpretable Decision Sets: A Joint Framework for Description and Prediction (2016, methodology): “Since our prediction tasks are multi-class, we used micro-averaged [14] AUC according to”
A systematic study of the class imbalance problem in convolutional neural networks (2017, methodology): “A more advanced approach was proposed by Hand and Till [106], that is M -measure.”
Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic (2003, background): “An alternative means of evaluating classifier performance that does not confound the false-positive and false-negative error rates is the Receiver Operating Characteristic or ROC curve (Green & Swets, 1966).”
Information, Divergence and Risk for Binary Experiments (2009, methodology): “…Operating Characteristic (ROC) curves, (Fawcett, 2006, 2004; Flach, 2003; Flach and Wu, 2005; Maxion and Roberts, 2004) the Area Under ROC Curve (AUC), (Cortes and Mohri, 2004; Hand, 2008; Hand and Till, 2001; Hanley and McNeil, 1982) and Cost Curves (Drummond and Holte, 2006; Torgersen, 1991).”
Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown (2003, background): “There are also analyses for when only a portion of the roc curve is of interest (McClish, 1989; Woods et al., 1997) and when analyzing more than two decisions (Swets & Pickett, 1982; Mossman, 1999; Hand & Till, 2001).”
Benchmarking and survey of explanation methods for black box models (2021, methodology): “The final score is obtained by taking the area under the curve (AUC) [62] of accuracy as a function of the percentage of removed pixels.”
Classification assessment methods (2020, background): “In multi-class classification problems, Provost and Domingos calculated the total AUC of all classes by generating a ROC curve for each class and calculate the AUC value for each ROC curve [10].”
Topic modelling for qualitative studies (2017, background): “AUC can be easily computed as where n pos and n neg are the number of positive and negative examples in the dataset, and P pos r i is the sum of the ranks of all positive examples [52].”
A Comparison of MCC and CEN Error Measures in Multi-Class Prediction (2010, background): “Although an agreed reasonably average-based build extension exists (presented in (Hand & Till, 2001)), several alternative formulations are being presented, either based on a multi-class ROC approximation (Everson & Fieldsend, 2006; Landgrebe & Duin, 2005, 2006, 2008)) or by viewing the ROC as a…”
Using AUC and accuracy in evaluating learning algorithms (2005, methodology): “Hand and Till [12] present a simple approach to calculating the AUC of a classifier below.”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, background): “In addition, formulations can be extended to the multi-class scenario following the one-vs-all or one-vs-one settings [62, 101, 186].”
ROC Graphs: Notes and Practical Considerations for Researchers (2007, background): “Hand and Till (2001) point out that Gini + 1 = 2 × AUC.”