AUC Optimization vs. Error Rate Minimization

By Corinna Cortes, Mehryar Mohri. 2003

692 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Developing an optimal short-form of the PTSD Checklist for DSM-5 (PCL-5) (2019, methodology): “We used inspection of the AUCs across models to narrow the range of short‐form scales in the validation sample (Cortez & Mohri, 2004).”
Learning Algorithms for the Classification Restricted Boltzmann Machine (2012, methodology): “The performance is evaluated in terms of retrieval performance using the area under the ROC curve (AROC) (Cortes and Mohri, 2004).”
A support vector method for multivariate performance measures (2005, methodology): “Also, methods for optimizing ROCArea have been proposed in the area of decision trees (Ferri et al., 2002), neural networks (Yan et al., 2003; Herschtal & Raskutti, 2004), boosting ( Cortes & Mohri, 2003; Freund et al., 1998), and SVMs (Herbrich et al., 2000; Rakotomamonjy, 2004).”
Supervised classification and mathematical optimization (2013, unknown): “See e.g. [69] for further details.”
Which method predicts recidivism best?: a comparison of statistical, machine learning and data mining predictive models (2013, methodology): “This led to the development of algorithms optimizing AUC directly (Cortes and Mohri, 2003; Clemençon and Vayatis, 2010).”
AUC-Maximized Deep Convolutional Neural Fields for Protein Sequence Labeling (2016, methodology): “In particular, (a) Cortes et al. [7] optimized AUC by RankBoost algorithm; (b) Ferri et al. [15] trained a decision tree by using AUC as splitting criteria; (c) Herschtal and Raskutti [21] trained a neural network by optimizing AUC; and (d) Joachims [23] proposed a generalized Support Vector…”
Precision-Recall versus Accuracy and the Role of Large Data Sets (2019, result): “Actually, results by Cortes and Mohri (2004) imply that precision-recall bounds are fundamentally different from AUC: for a ﬁxed class imbalance and accuracy, various ranking functions may feature substantially different AUC scores, in contrast to precision-recall bounds.”
Bipartite Ranking through Minimization of Univariate Loss (2011, background): “Focusing on loss minimization in this paper, we shall mostly refer to a reciprocal measure, namely the rank loss defined as 1−AUC.”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, methodology): “The RankBoost algorithm was later applied to AUC maximization [28].”
Margin-based Ranking and an Equivalence between AdaBoost and RankBoost (2009, methodology): “A number of papers, including those of Cortes and Mohri (2004) and Caruana and Niculescu-Mizil (2006) have noted that in fact, AdaBoost experimentally seems to be very good at the bipartite ranking problem, even though it was RankBoost that was explicitly designed to solve this problem, not…”
Chromatic PAC-Bayes Bounds for Non-IID Data (2009, methodology): “…function f ∈ R X such that h ( X, X ′ ) = f ( X ) − f ( X ′ ), in which case (overloading notation): where we recognize in ˆ R Auc ( f, S ) one minus the Area under the Roc curve, or Auc , of f on S (Agarwal et al., 2005; Cortes and Mohri, 2004), hence the Auc superscript in the name of the risk.”
On multi-class classification through the minimization of the confusion matrix norm (2013, background): “Meanwhile, the correlations between these alternative measures and accuracy have been partly studied (Cortes and Mohri, 2004) without any theoretical result so far (He and Garcia, 2009).”
Surrogate regret bounds for bipartite ranking via strongly proper losses (2012, methodology): “Nevertheless, it has often been observed that algorithms such as AdaBoost, logistic regression, and in some cases even SVMs, which minimize the exponential, logistic, and hinge losses respectively in the standard (non-pairwise) setting, also yield good bipartite ranking performance [11, 20, 24].”
Scalable Learning of Non-Decomposable Objectives (2016, unknown): “Unfortunately, this discrepancy can lead to inferior results, as is illustrated in Figure 1 (see also, for example, [7, 9, 27]).”
An experimental comparison of performance measures for classification (2009, background): “However, little is known when the performance measure is different from accuracy (Freund et al., 2003; Cortes and Mohri, 2003; Lebanon and Lafferty, 2002).”
AUCpreD: proteome-level protein disorder prediction by AUC-maximized deep convolutional neural fields (2016, background): “AUC is insensitive to changes in class distribution (Cortes and Mohri, 2004) because that the ROC curve specifies the relationship between false positive (FP) rate and true positive (TP) rate, which are independent of class distribution (Fawcett, 2004).”