The language-as-fixed-effect fallacy: A critique of language statistics in psychological research.

By Herbert H. Clark. 1973

Subfields: Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Science & Psychology

Citations in the corpus

Can language models learn from explanations in context? (2022, background): “Failing to model these factors leads to issues such as the “stimulus-as-ﬁxed-effect fallacy” (Clark, 1973)—mistaking a stimulus or dataset effect for a more general principle.”
To transform or not to transform: using generalized linear mixed models to analyse reaction time data (2015, unknown): “…the mean, and many have begun to turn to the multilevel properties of LMMs as a way of simultaneously controlling for (or explicitly investigating) individual sensitivity between each item or participant as an explanation of overall diﬀerences between conditions (Clark, 1973; Locker et al., 2007).”
Violent Video Games and Hostile Expectations: A Test of the General Aggression Model (2002, background): “This problem is not unique to media violence research, of course (Clark, 1973; Wells & Windschitl, 1999).”
The potentiation of grasp types during visual object categorization (2001, methodology): “For the effects of most relevance to the hypotheses under consideration we give both analyses by participants as well as by objects and include a calculation of Min F¢ (Clark, 1973) to provide a conservative test of the ability of the effect to generalise to a new sample of participants and objects.”
The Inference of Perceived Usability From Beauty (2010, background): “This double requirement has been widely used in psychology since it was suggested in the area of psycholinguistics by Clark (1973), and many psychology journals make a point of requiring authors to establish generalizability across participants and items, where appropriate (see, e.g., the Editorial…”
Exploring the Articulatory Loop (1984, methodology): “the evidence obtained by Colle and Welsh (1 976) and Salame and Baddeley (1982) on the effects of irrelevant speech on memory and was further supported by evidence from the detailed investigation of a patient suffering from defective short-term memory (Vallar and Baddeley, in press).”
The effect of word predictability on reading time is logarithmic (2013, unknown): “This issue is what motivates the use of by-participant and by-item ‘‘random effects’’ in repeated-measures ANOVA and mixed-effects regression models (Baayen, Davidson, & Bates, 2008; Clark, 1973; see also Barr, Levy, Scheepers, & Tily (2013) for specific discussion on this issue).”
Default Bayes factors for ANOVA designs (2012, background): “Clark (1973) noted that averaging treats items as fixed rather than as random effects, and the correlation in performance across items leads to downward bias in the estimate of residual variability.”
The Effects of Interactive Latency on Exploratory Visual Analysis (2014, result): “However, treating the scenario as a fixed effect limits our ability to generalize the results to analysis scenarios not included in the study [14].”
The Use of Media in Media Psychology (2016, background): “A debate in the 1980s had origins in an influential critique of linguistics research that identified the fallacy of considering language stimuli as a fixed effect (Clark, 1973); that is, language stimuli in experiments usually do not represent all possible variations (or levels) but rather a random ”
Political Ideology at Face Value (2010, methodology): “To increase the generalizability of the findings, we analyzed all data with targets (averaging over participants) and with participants (averaging over targets) as the units of analysis (see Clark, 1973).”
Weak quantitative standards in linguistics research. (2010, background): “Multiple instances of the relevant construction are needed to evaluate whether an observed effect generalizes across different sets of lexical items [7].”
Do all ducks lay eggs? The generic overgeneralization effect (2011, methodology): “As is conventional, we use F 1 and F 2 to report participant and item analyses respectively , as well as min F 0 (Clark, 1973).”
When Semantics Meets Phonetics: Acoustical Studies of Second-Occurrence Focus (2007, methodology): “Following Clark (1973) and Raaijmakers et al. (1999), a minimum F was calculated from the F1 and F2 analyses.”