Torturing data for the sake of generality: How valid are our regression models?

By Julian D. Olden, Donald A. Jackson. 2000

152 citations recorded by the source corpus.

Subfields: Ecological Modeling, Environmental Chemistry, Nature and Landscape Conservation

View Paper: https://www.semanticscholar.org/paper/fe45d1a3b0e54dfe9ff4076d40532bca8d3a4895

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecological Modeling

In Environmental Chemistry

In Nature and Landscape Conservation

Citations in the corpus

Quantifying variable importance in a multimodel inference framework (2016, methodology): “…the challenges to drawing inferences from over-parameterized global regression models or models selected from arbitrary stepwise approaches (e.g. Olden & Jackson 2000; Whittingham et al. 2006), an IT approach is well suited to studying complex ecological systems as it allows for the…”
Influences of Temperature and Environmental Variables on the Distribution of Bull Trout within Streams at the Southern Margin of Its Range (2003, methodology): “Cross validations were performed both within and between data sets to evaluate model predictions (Olden and Jackson 2000).”
On the analysis of non‐linear allometries (2009, unknown): “Many solutions to the problem of choosing between different statistical models have been suggested: see, for example Burnham and Anderson (2002); Johnson and Omland (2004) and Olden and Jackson (2000) .”
Influences on the transport and establishment of exotic bird species: an analysis of the parrots (Psittaciformes) of the world (2004, methodology): “A manual selection method was used to assess and remove nonsignificant terms manually from multivariate models until no further terms could be removed without reducing the model’s explanatory powers (Olden & Jackson, 2000).”
The effect of prevalence and its interaction with sample size on the reliability of species distribution models (2009, methodology): “In the absence of a general rule, it is generally assumed that the greater the sample size, the more accurate the model (Cumming 2000, Olden and Jackson 2000, McPherson et al. 2004, Martínez-Meyer 2005, Reese et al. 2005).”
What controls who is where in freshwater fish communities the roles of biotic, abiotic, and spatial factors (2001, methodology): “Being able to correctly identify the communities at a series of sites provides an excellent test of our understanding, provided such tests are implemented correctly (e.g., using appropriate methods to test and validate our models; see Olden and Jackson 2000).”
Assessing transferability of ecological models: an underappreciated aspect of statistical validation (2012, methodology): “We expect this to be a general rule and to be true regardless of the size of the data set, which stands in contrast to random cross-validation, in which the number of groups becomes irrelevant as the size of the data set grows arbitrarily large (Olden & Jackson 2000).”
Applied Predictive Modeling (2013, background): “The second issue has been extensively studied (Derksen and Keselman 1992; Olden and Jackson 2000; Harrell 2001; Whittingham et al. 2006).”
Coefficient shifts in geographical ecology: an empirical evaluation of spatial and non-spatial regression (2009, background): “Thus, we add one more item to the long list of problems related to the use of multiple regression in ecology (Olden and Jackson 2000, Graham 2003, Grace and Bollen 2005, Whittingham et al. 2006 and references therein).”
Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks (2002, methodology): “N-fold cross validation was used to determine optimal network design as it provides a nearly unbiased estimate of prediction success (Olden and Jackson, 2000).”
Five (or so) challenges for species distribution modelling (2006, methodology): “…the future use a simple form of verification (the resubstituition approach) in which the same data used for model calibration are used to test the models (Araújo et al., 2005a); a similar observation was documented for studies modelling current species distributions (e.g. Olden & Jackson, 2000).”
Determinants of local spider (Araneidae and Thomisidae) species richness on a regional scale: climate and altitude vs. habitat structure (2007, background): “Additionally, although statisticians emphasise that a model without proper validation has no merit ( Olden & Jackson, 2000 ), no validation of models is reported in any of these two formerly cited studies.”
Accurately Defining the Reference Condition for Summary Biotic Metrics: A Comparison of Four Approaches (2006, methodology): “We also externally validated the models with an independent data set (the validation sites) to give a more realistic estimate of prediction error (Olden & Jackson, 2000).”
An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data (2004, background): “Examples of simulation studies in ecology that compared different statistical methodologies include, but are not limited to, timeseries analysis (Berryman and Turchin, 2001), ordination (Minchin, 1987; Jackson, 1993), regression analysis (Olden and Jackson, 2000), type I and II error analysis (Peres”