AUC: a misleading measure of the performance of predictive distribution models

By Jorge M. Lobo, A. Jiménez‐Valverde, Raimundo Real. 2008

3,478 citations recorded by the source corpus.

Subfields: Epidemiology, Global and Planetary Change, Safety, Risk, Reliability and Quality

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Epidemiology

In Global and Planetary Change

In Safety, Risk, Reliability and Quality

Citations in the corpus

Machine Learning that Matters (2012, background): “…several drawbacks, including summarizing performance over all possible regimes even if they are unlikely ever to be used (e.g., extremely high false positive rates), and weighting false positives and false negatives equally, which may be inappropriate for a given problem domain (Lobo et al., 2008).”
Comparing sample bias correction methods for species distribution modeling using virtual species (2021, background): “…on variable importance and provides a measure of a model’s ability to discriminate between presence and absence localities, independent of an arbitrary cutoff threshold (Cumming 2000), although it has been criticized for its sensitivity to geographic extent, among other factors (Lobo et al. 2008).”
Glacial refugia of temperate trees in Europe: insights from species distribution modelling (2008, background): “dependence on study area extent and prevalence (Lobo et al. 2008).”
Invasive weed species’ threats to global biodiversity: Future scenarios of changes in the number of invasive species in a changing climate (2020, background): “Although AUC is a reliable measure of discrimination ability, it presents limitations for ecological realism in modeled distributions where estimations are based only on presence data (Jiménez-Valverde, 2014; Lobo et al., 2008).”
The importance of correcting for sampling bias in MaxEnt species distribution models (2013, methodology): “In this respect, the most commonly used metric for model quality (test AUC) can be misleading (Lobo et al., 2007) and can poorly represent the underlying biology of the system being studied (Warren & Seifert, 2011).”
Impact of the quality of climate models for modelling species occurrences in countries with poor climatic documentation: a case study from Bolivia (2010, background): “Although this question is beyond the scope of this study, it is evident that modelled ranges with high AUC values may still be unreliable (see also Lobo et al., 2008; Peterson et al., 2008).”
Spatial bias in the GBIF database and its effect on modeling species' geographic distributions (2014, methodology): “…by modelling extent, for being not consistent with other evaluation criteria, and (if applied to presence-only data) for not representing “true” AUC (Lobo et al., 2008; Peterson et al., 2008; Jiménez-Valverde, 2011; Barve et al., 2011; L. Ballesteros-Mejia, I.J. Kitching & J. Beck, unpubl.).”
Pest Risk Maps for Invasive Alien Species: A Roadmap for Improvement (2010, neutral): “Generally, statistical tests of model performance are preferred to qualitative comparisons of model output and known occurrences, but there are limitations to many goodness-of-fit measures (Lobo et al. 2008, Peterson et al. 2008).”
Rethinking receiver operating characteristic analysis applications in ecological niche modeling (2008, methodology): “ROC can be applied directly to evaluation of SDM predictions (Fielding and Bell, 1997; Fawcett, 2003; Phillips et al., 2006), although even this functionality is not above question (Lobo et al., 2007).”
High uncertainty in the effects of data characteristics on the performance of species distribution models (2021, methodology): “…studied region (Relative Occurrence Area, ROA) determines predictive success, as the rate of well-predicted absences increases when the extent of analyses is much larger than the species distribution, hence inflating performance metrics (Lobo, 2008; Lobo et al., 2008; Moudrý and Šímová, 2013).”
The importance of biotic interactions in species distribution models: a test of the Eltonian noise hypothesis using parrots (2014, background): “Many studies have pointed out problems with the AUC approach in SDM, but these problems mostly concern the comparison of different methods and/or species, which is not the case here (Raes & ter Steege, 2007; Lobo et al., 2008; Peterson et al., 2008).”