Testing the ability of species distribution models to infer variable importance

By Adam B. Smith, Maria João Santos. bioRxiv, 2019

87 citations recorded by the source corpus.

Identifying biophysical factors that define species’ niches and influence geographical ranges is a fundamental pursuit of ecology. Frequently, models of species’ distributions or niches are used to infer the importance of range- and niche-defining variables. However, very few—if any—studies examine how reliably distribution and niche models can be used for inference. Here we use a simulation approach to understand the conditions under which species distribution models reliably measure variable importance. Using a set of scenarios of increasing complexity, we explore how well models can be used to 1) discriminate between variables that vary in importance and 2) calibrate the effect of variables relative to an “omniscient” model used to simulate the species. Variable importance was assessed using a sensitivity test in which each predictor was permuted in turn. Importance was inferred by comparing model performance between permuted and unpermuted predictions and by calculating the correlation between permuted and unpermuted predictions. Of five metrics of importance (correlation statistic and AUC each calculated with presences/absences or presences/background sites, plus the Continuous Boyce Index), only the Continuous Boyce Index was capable of indicating absolute (versus relative) variable importance. In simple scenarios with one influential environmental variable with a linear spatial gradient and one uninfluential randomly-distributed variable, models were unable to discriminate reliably between variables under conditions that are typically challenging (low sample size, high prevalence, small spatial extent, coarse spatial data resolution with low spatial autocorrelation, and high collinearity between variables). In more complex scenarios with two influential environmental variables, models successfully discriminated between variables when they acted unequally, but overestimated the importance of the stronger variable and underestimated the importance of the weaker variable. When variables had equal influence, models underestimated importance when niche breadth was narrow. Generalized additive models and Maxent had better discrimination accuracy than boosted regression trees. Our work demonstrates that permutation tests can reliably discriminate between variables with different levels of influence but cannot accurately measure the magnitude of influence. The frequency with which distribution and niche models are used to identify influential variables begs further research into methods for assessing variable importance.

Subfields: Ecological Modeling, Ecology, Nature and Landscape Conservation

View Paper: https://www.semanticscholar.org/paper/d2e4aa23ee8e16a6e9633b03a626aef87996b95f