Mixed Effects Models are Sometimes Terrible

By Christopher Eager, Joseph Roy. 2017

63 citations recorded by the source corpus.

Mixed-effects models have emerged as the gold standard of statistical analysis in different sub-fields of linguistics (Baayen, Davidson & Bates, 2008; Johnson, 2009; Barr, et al, 2013; Gries, 2015). One problematic feature of these models is their failure to converge under maximal (or even near-maximal) random effects structures. The lack of convergence is relatively unaddressed in linguistics and when it is addressed has resulted in statistical practices (e.g. Jaeger, 2009; Gries, 2015; Bates, et al, 2015b) that are premised on the idea that non-convergence is an indication that a random effects structure is over-specified (or not parsimonious), the parsimonious convergence hypothesis (PCH). We test the PCH by running simulations in lme4 under two sets of assumptions for both a linear dependent variable and a binary dependent variable in order to assess the rate of non-convergence for both types of mixed effects models when a known maximal effect structure is used to generate the data (i.e. when non-convergence cannot be explained by random effects with zero variance). Under the PCH, lack of convergence is treated as evidence against a more maximal random effects structure, but that result is not upheld with our simulations. We provide an alternative model, fully specified Bayesian models implemented in rstan (Stan Development Team, 2016; Carpenter, et al, in press) that removed the convergence problems almost entirely in simulations of the same conditions. These results indicate that when there is known non-zero variance for all slopes and intercepts, under realistic distributions of data and with moderate to severe imbalance, mixed effects models in lme4 have moderate to high non-convergence rates which can cause linguistic researchers to wrongfully exclude random effect terms.

Subfields: Economics and Econometrics, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8a128765b173d90c719411b7ed4eaac7cb7bcfa7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

An Introduction to Mixed Models for Experimental Psychology (2019, unknown): “Additionally, speciﬁcally binomial GLMMs can be quite prone to producing singular ﬁts or other convergence problems due to the limited amount of information provided by each data point (i.e., 0 or 1; see Eager & Roy, 2017).”
Best practice guidance for linear mixed-effects models in psychological science (2019, methodology): “…et al, 2017), using Principal Components Analysis to determine the most meaningful slopes (Bates et al., 2015), switching to alternate optimization algorithms (see comments by Bolker, 2015), or indeed to alternate programming languages or approaches (e.g., Bayes estimation, Eager & Roy, 2017).”