Model selection and psychological theory: A discussion of the differences between the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC)

By Scott Vrieze. Psychological methods, 2012

1,786 citations recorded by the source corpus.

This article reviews the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) in model selection and the appraisal of psychological theory. The focus is on latent variable models given their growing use in theory testing and construction. We discuss theoretical statistical results in regression and illustrate more important issues with novel simulations involving latent variable models including factor analysis, latent profile analysis, and factor mixture models. Asymptotically, the BIC is consistent, in that it will select the true model if, among other assumptions, the true model is among the candidate models considered. The AIC is not consistent under these circumstances. When the true model is not in the candidate model set the AIC is effcient, in that it will asymptotically choose whichever model minimizes the mean squared error of prediction/estimation. The BIC is not effcient under these circumstances. Unlike the BIC, the AIC also has a minimax property, in that it can minimize the maximum possible risk in finite sample sizes. In sum, the AIC and BIC have quite different properties that require different assumptions, and applied researchers and methodologists alike will benefit from improved understanding of the asymptotic and finite-sample behavior of these criteria. The ultimate decision to use AIC or BIC depends on many factors, including: the loss function employed, the study's methodological design, the substantive research question, and the notion of a true model and its applicability to the study at hand.

Subfields: Computer Networks and Communications, Experimental and Cognitive Psychology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e1cb36d6209638b9478ed01e7944022a6dba06d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Experimental and Cognitive Psychology

In Statistics and Probability

Citations in the corpus

COVID-19 vaccine hesitancy in the UK: the Oxford coronavirus explanations, attitudes, and narratives survey (Oceans) II (2020, background): “Individuals rate on a scale from almost everyday (1) to never (6) how often nine negative experiences happen to them (Williams et al., 1997).”
Sensitivity and Specificity of Information Criteria (2018, neutral): “However, the real situation is more com270 plicated; AIC is not a defective BIC, nor vice versa (see Kieseppä, 2003; Shibata, 271 1981, 1986; Pötscher, 1991; Vrieze, 2012).”
Choosing prediction over explanation in psychology: Lessons from machine learning (2017, methodology): “A somewhat traditional way to compare nonnested models is to rely on the Aikake Information Criterion (AIC) or Bayesian Information Criterion (BIC), but these are subject to a variety of somewhat complicated assumptions (Burnham & Anderson, 2004; Vrieze, 2012) and in any case can only be computed fo”
Model Comparison and the Principle of Parsimony (2015, background): “In fact, there is cause to believe that the AIC tends to select complex models that overfit the data (O’Hagan & Forster, 2004; for a discussion see Vrieze, 2012).”
The Structure of Counterproductive Work Behavior (2016, neutral): “AIC and BIC are based on different mathematical models and were found to have different properties and largely opposing pros and cons under a variety of conditions (Vrieze, 2012).”
Practitioner’s Guide to Latent Class Analysis: Methodological Considerations and Common Pitfalls. (2020, methodology): “BIC tends to perform poorly when the sample size is modest because of a high probability of extracting too few classes and may be outperformed by the AIC (52).”
Maturational changes in frontal EEG alpha and theta activity from infancy into early childhood and the relation with self-regulation in boys and girls (2024, methodology): “Lower AIC and BIC values suggest a better fit to the data, with decreases in values larger than 10 indicating a serious improvement in model fit (Raftery, 1995; Vrieze, 2012).”