Categorical Data Analysis: Away from ANOVAs (transformation or not) and towards Logit Mixed Models

By T. Florian Jaeger, Cognitive Sciences. Journal of Memory and Language, 2008

3,410 citations recorded by the source corpus.

This paper identifies several serious problems with the widespread use of ANOVAs for the analysis of categorical outcome variables such as forced-choice variables, question-answer accuracy, choice in production (e.g. in syntactic priming research), et cetera. I show that even after applying the arcsine-square-root transformation to proportional data, ANOVA can yield spurious results. I discuss conceptual issues underlying these problems and alternatives provided by modern statistics. Specifically, I introduce ordinary logit models (i.e. logistic regression), which are well-suited to analyze categorical data and offer many advantages over ANOVA. Unfortunately, ordinary logit models do not include random effect modeling. To address this issue, I describe mixed logit models (Generalized Linear Mixed Models for binomially distributed outcomes, Breslow & Clayton, 1993), which combine the advantages of ordinary logit models with the ability to account for random subject and item effects in one step of analysis. Throughout the paper, I use a psycholinguistic data set to compare the different statistical methods.

Subfields: Artificial Intelligence, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/00faba344d392617b93e67430f874ba57e04e994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Comprehension of reflexive and personal pronouns in children with autism: A syntactic or pragmatic deficit? (2012, methodology): “Mixed logistic regression models have been argued to be more suitable for analyzing data in psycholinguistic research than commonly used repeated measures analyses of variance (e.g., Jaeger, 2008; Quene & van den Bergh, 2008).”
Mixed effect models for genetic and areal dependencies in linguistic typology (2011, background): “For each of these three possible genealogical grouping factors, we can ask whether its inclusion in the model is statistically justified.”
The Utility and Application of Mixed-Effects Models in Second Language Research (2015, methodology): “Logit mixedeffects models for example can be used to analyse data with a binomial dependent variable, such as a binary grammaticality judgement (see Jaeger, 2008 for review).”
Predicting syntax: Processing dative constructions in American and Australian varieties of English (2010, methodology): “To attack this problem head-on, we make use of multilevel (also termed mixed effect) regression models, which are of growing importance in the language sciences (Baayen 2008, Johnson 2008, Baayen et al. 2008, Quené and van den Bergh 2008, Jaeger 2008).”
Crosslinguistic influence in the acquisition of a third language: The Linguistic Proximity Model (2017, methodology): “We fitted a generalized linear mixed effects logistic regression (e.g. Jaeger, 2008; Quené & van der Bergh, 2008) to estimate the effects of condition (Adv-V, Aux-S), language (Norwegian, Norwegian-Russian, Russian) and their interaction on the correctness of judgments (1 if the sentence was judged…”
Effects of written corrective feedback and language aptitude on verb tense accuracy (2019, methodology): “…using mixed-effects logit models in R (R Development Core Team, 2011) with the R package lme4 (Bates, Maechler, & Bolker, 2011), as mixed-effects logit models allow every observation to be modeled while accounting for the fact that there are many observations for each participant (Jaeger, 2008).”
The virulence of entomopathogenic fungi against Bemisia tabaci biotype B (Hemiptera: Aleyrodidae) and their conidial production using solid substrate fermentation (2013, positive): “, residual deviance greater than 1), it was included in model as another random effect (Jaeger, 2008).”
The nature and specificity of paired associate learning deficits in children with dyslexia (2014, background): “Second, ANOVA can lead to spurious results when the dependent variable constitutes a binary outcome, even when classic transformations have been applied (Jaeger, 2008).”
Explicit and Implicit Lexical Knowledge: Acquisition of Collocations Under Different Input Conditions (2013, methodology): “…pp. 121–159 134 Third, it can handle interval-scale (e.g., reaction time) measures through linear mixed-effects (LME; see Baayen, 2008) models as well as categorical (e.g., correct versus incorrect) measures using mixed logit models (the LME alternative for categorical data; see Jaeger, 2008).”
Analyzing Reaction Times (2010, methodology): “In what follows, we discuss, a large dataset illustrating some of the novel possibilities offered by the mixed-modeling framework building on prior introductions (here we build on prior introductions given by Pinheiro & Bates, 2000; Baayen et al., 2008; Jaeger, 2008; Quené & Bergh, 2008, etc.).”
'And' or 'or': General use coordination in ASL (2013, methodology): “4.3 Results Results were analyzed using mixed logit models, which are well-suited for modeling data like the binary felicity judgements that the participants made in this study (Baayen 2007, Jaeger 2008).”
Maternal Responsiveness and the Development of Directed Vocalizing in Social Interactions (2014, methodology): “The model that was fitted is a logistic model (see Jaeger, 2008).”