Using AUC and accuracy in evaluating learning algorithms

By Jin Huang, Charles X. F. Ling. IEEE Transactions on Knowledge and Data Engineering, 2005

2,138 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/85d727b119304dde458bcd8cf5cb87a906fb41ba

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

Word2Vec inversion and traditional text classifiers for phenotyping lupus (2017, methodology): “Bootstrap resampling, randomly selecting a sample to add to the pipeline from the class with less instances was used to achieve a balanced dataset with an equal number of patients and controls in both training and testing data sets [16].”
Robust Graph Representation Learning via Neural Sparsification (2020, methodology): “Due to the imbalanced classes in the Transaction dataset, models are evaluated with AUC value (Huang & Ling, 2005).”
Study of the impact of resampling methods for contrast pattern based classifiers in imbalanced databases (2016, methodology): “Finally, based on the runtime and AUC results, we can say that RUS, TL and SMOTE-TL are the resampling methods with the best trade-off between runtime and AUC results.”
A Pattern-Based Approach for Detecting Pneumatic Failures on Temporary Immersion Bioreactors (2019, background): “Abbreviations The following abbreviations are used in this manuscript: AB Adaptive Boosting AUC Area Under the receiver operating characteristic Curve B-TPMiner Bagging-TPMiner CP Contrast Pattern DB Database kNN K-Nearest Neighbor MLP Multilayer Perceptron NBayes Naïve Bayes LogReg Logistic Regress”
Contrast Pattern-Based Classification for Bot Detection on Twitter (2019, methodology): “Bot or Not uses a Random Forest classifier [35] over 1,000 features from all the categories, obtaining a 0.95 of Area Under the ROC Curve (AUC).”
A comparative study of machine learning classifiers for modeling travel mode choice (2017, background): “…of land use classes green Proportion of green space per post code area in % Weather precip Daily precipitation sum in mm temp Daily maximum temperature in ◦C wind Daily average wind speed in m/s AC CE PT ED M AN US CR IP T be competitive with more advanced classifiers (e.g. Huang and Ling, 2005).”
Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics (2012, methodology): “The Area Under the ROC Curve (AUC) (Huang & Ling, 2005) corresponds to the probability of correctly identifying which of the two stimuli is noise and which is signal plus noise.”
A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability (2009, neutral): “Well-known accuracy measures for binary-class problems are: classification rate, precision, sensitivity, specificity, G-mean (Barandela et al. 2003), F-score, AUC (Huang and Ling 2005), Youden's index c (Youden 1950) and Cohen's Kappa (Ben-David 2007).”
Boosting methods for multi-class imbalanced data classification: an experimental review (2020, methodology): “Furthermore, the degree of Indif-ferency of metrics is tested as proposed in [22].”
Extreme Value Theory for Open Set Classification - GPD and GEV Classifiers (2018, methodology): “Also here, we choose to evaluate the performance of the different algorithms using the obtained AUC that does not require to ﬁx a threshold, instead of using the F -measure [27] as in the original paper.”
Structural and functional biomarkers of prodromal Alzheimer's disease: A high-dimensional pattern classification study (2008, background): “This process allows us to determine the optimal parameters of the classifier, including the number of clusters that is used, so that the area under the ROC curve is maximized (Huang and Ling, 2005).”
An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics (2013, methodology): “The Area Under the ROC Curve (AUC) [70] corresponds to the probability of correctly identifying which one of the two”
Text Classification Algorithms: A Survey (2019, methodology): “text classiﬁcation methods. There are many methods available for evaluating supervised techniques. Accuracy calculation is the simplest method of evaluation, but does not work for unbalanced datasets [28]. In Section5, we outline the following evaluation methods for text classiﬁcation algorithms: Fb”
An experimental comparison of performance measures for classification (2009, methodology): “However, using exactly the same algorithms, in (Huang and Ling, 2005) the authors show that Naive Bayes is significantly better than pruned decision trees in AUC.”