Why do we still use stepwise modelling in ecology and behaviour?

By Mark J. Whittingham, Philip Stephens, R. Bradbury, R. Freckleton. Journal of Animal Ecology, 2006

1,544 citations recorded by the source corpus.

1. The biases and shortcomings of stepwise multiple regression are well established within the statistical literature. However, an examination of papers published in 2004 by three leading ecological and behavioural journals suggested that the use of this technique remains widespread: of 65 papers in which a multiple regression approach was used, 57% of studies used a stepwise procedure. 2. The principal drawbacks of stepwise multiple regression include bias in parameter estimation, inconsistencies among model selection algorithms, an inherent (but often overlooked) problem of multiple hypothesis testing, and an inappropriate focus or reliance on a single best model. We discuss each of these issues with examples. 3. We use a worked example of data on yellowhammer distribution collected over 4 years to highlight the pitfalls of stepwise regression. We show that stepwise regression allows models containing significant predictors to be obtained from each year's data. In spite of the significance of the selected models, they vary substantially between years and suggest patterns that are at odds with those determined by analysing the full, 4-year data set. 4. An information theoretic (IT) analysis of the yellowhammer data set illustrates why the varying outcomes of stepwise analyses arise. In particular, the IT approach identifies large numbers of competing models that could describe the data equally well, showing that no one model should be relied upon for inference.

Subfields: Ecological Modeling, Economics and Econometrics, Nature and Landscape Conservation

View Paper: https://www.semanticscholar.org/paper/f4a291d83e599562264760891c6fdd8007b36270

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecological Modeling

In Economics and Econometrics

In Nature and Landscape Conservation

Citations in the corpus

Assessing road effects on bats: the role of landscape, road features, and bat activity on road-kills (2013, neutral): “However, the significance levels of the variables were presented and discussed because, despite Burnham and Anderson (2002) do not recommend mixing information-theoretic model comparison (ITMC) and nullhypothesis theory, it is possible and helpful to use the later to enhance the confidence in the co”
Ecological Models and Data in R (2008, background): “Blind fitting of models with many covariates can get you in trouble (Whittingham et al., 2006).”
Female–female aggression around mating: an extra cost of sociality in a multimale primate society (2011, methodology): “The significance of the fixed variables was always tested using the full model (i.e., inferences were drawn with all predictors present) to avoid the potential problems associated with stepwise model selection procedures (Whittingham et al. 2006; Mundry and Nunn 2009).”
A brief introduction to mixed effects modelling and multi-model inference in ecology (2018, background): “NHST and stepwise deletion have come under heavy criticism; they can overestimate the effect size of ‘significant’ predictors (Whittingham et al 2006; Forstmeier & Schielzeth 2011) and force the researcher to focus on a single best model as if it were the only combination of predictors with support…”
Information-theoretic approaches to statistical analysis in behavioural ecology: an introduction (2010, methodology): “P<0.05) as a criterion for the exclusion or inclusion of terms, but some users apply criteria other than probability levels of significance (even AIC, against which, see warnings in Whittingham et al. 2006).”
Model averaging, missing data and multiple imputation: a case study for behavioural ecology (2010, methodology): “The reason for this has been an increased focus on modelling of data, rather than binary hypothesis testing (Hilborn and Mangel 1997; Bolker 2008), and the consequence has been a sharpening and refocusing of statistical practice and philosophy among ecologists and evolutionary biologists, even if no”
Comparison of stepwise selection and Bayesian model averaging for yield gap analysis (2008, background): “Whittingham et al. (2006) have reviewed 508 papers published ∗ Corresponding author.”
The consequences of checking for zero‐inflation and overdispersion in the analysis of count data (2019, background): “In ecology, some have warned about model selection bias (e.g. Buckland et al., 1997), but the problem ‘remains widely over- looked’ (Whittingham et al., 2006).”
Plant species identity and arbuscular mycorrhizal status modulate potential nitrification rates in nitrogen‐limited grassland soils (2011, methodology): “no autocorrelation (Whittingham et al. 2006).”
Eight (and a half) deadly sins of spatial analysis (2012, methodology): “…pitfalls associated with multiple regression in an ecological context and to propose methods to deal with them (Freckleton, 2002, 2009; Graham, 2003; Whittingham et al., 2006), it remains that while it is a reasonable method for predicting relationships, it is a very poor one for explaining them.”
Developing multiple hypotheses in behavioral ecology (2010, background): “Whittingham et al. (2006) fit all possible models (with the exception that they did not include interactions) to data for yellowhammer distributions, resulting in over 500 possible models.”