Variable selection – A review and recommendations for the practicing statistician

By Georg Heinze, C. Wallisch, Daniela Dunkler. 2018

1,370 citations recorded by the source corpus.

Statistical models support medical research by facilitating individualized outcome prognostication conditional on independent variables or by estimating effects of risk factors adjusted for covariates. Theory of statistical models is well‐established if the set of independent variables to consider is fixed and small. Hence, we can assume that effect estimates are unbiased and the usual methods for confidence interval estimation are valid. In routine work, however, it is not known a priori which covariates should be included in a model, and often we are confronted with the number of candidate variables in the range 10–30. This number is often too large to be considered in a statistical model. We provide an overview of various available variable selection methods that are based on significance or information criteria, penalized likelihood, the change‐in‐estimate criterion, background knowledge, or combinations thereof. These methods were usually developed in the context of a linear regression model and then transferred to more generalized linear models or models for censored survival data. Variable selection, in particular if used in explanatory modeling where effect estimates are of central interest, can compromise stability of a final model, unbiasedness of regression coefficients, and validity of p‐values or confidence intervals. Therefore, we give pragmatic recommendations for the practicing statistician on application of variable selection methods in general (low‐dimensional) modeling problems and on performing stability investigations and inference. We also propose some quantities based on resampling the entire variable selection process to be routinely reported by software packages offering automated variable selection algorithms.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Tammy Jiang
2. J. Gradus
3. A. Rosellini
4. Mark S. Gilthorpe
5. G. Ellison
6. M. Fox
7. Nan Liu
8. P. Tennant

Citations in the corpus

• AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, methodology): “Although users may benefit from the built-in automation of AutoScore for developing a clinical score, domain knowledge is equally important in building the scoring models, as suggested in many studies [10,51].”
• Supervised machine learning: A brief primer (2020, methodology): “Reducing the number of variables reduces the dimensionality of the data, and several approaches exist (e.g., for a review see Heinze et al., 2018).”
• Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations (2020, background): “Many approaches are available to assist with deciding which variables to adjust for from a list of potential candidates, including various theory-free statistical criteria and algorithms.(5) Unfortunately, few of these conventional approaches explicitly consider the role of each variable in relation”