Methodological Urban Legends: The Misuse of Statistical Control Variables

By Paul E. Spector, Michael T. Brannick. 2011

1,127 citations recorded by the source corpus.

The automatic or blind inclusion of control variables in multiple regression and other analyses, intended to purify observed relationships among variables of interest, is widespread and can be considered an example of practice based on a methodological urban legend. Inclusion of such variables in most cases implicitly assumes that the control variables are somehow either contaminating the measurement of the variables of interest or affecting the underlying constructs, thus distorting observed relationships among them. There are, however, a number of alternative mechanisms that would produce the same statistical results, thus throwing into question whether inclusion of control variables has led to more or less accurate interpretation of results. The authors propose that researchers should be explicit rather than implicit regarding the role of control variables and match hypotheses precisely to both the choice of variables and the choice of analyses. The authors further propose that researchers avoid testing models in which demographic variables serve as proxies for variables that are of real theoretical interest in their data.

Subfields: Economics and Econometrics, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/58d9a442e6ec67ec5ca27d96814660ad58f8b9af

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Sociology and Political Science

Citations in the corpus

Office Chit-Chat as a Social Ritual: The Uplifting Yet Distracting Effects of Daily Small Talk at Work (2020, result): “Importantly, our results hold with or without controls; we retained them as a more conservative examination (Spector & Brannick, 2011).”
Consequences of knowledge hiding: The differential compensatory effects of guilt and shame (2018, neutral): “Following the recommendations of Spector and Brannick (2011), we thus report our results without control variables.”
Perceptions of high‐involvement work practices and burnout: the mediating role of job demands (2016, methodology): “The nonsignificant controls were omitted from further analysis to avoid misinterpretation of the results (Spector and Brannick, 2011).”
Service Employee Responses to Angry Customer Complaints (2017, background): “Therefore, we report in the remainder of this analysis the findings without controls to avoid issues related to reduced statistical power and potential spurious suppression that might be introduced by the controls (cf. Carlson and Wu 2012; Spector and Brannick 2011).”
When do emotionally exhausted employees speak up? Exploring the potential curvilinear relationship between emotional exhaustion and voice (2014, background): “We measured gender, age, education level, industrial tenure, and relation with supervisor because of their potential impact on prohibitive voice behaviors (Spector & Brannick, 2011).”
The Illusion of Statistical Control (2012, methodology): “In purification (see Spector & Brannick, 2011), it is assumed that measures of one or more IVs and/or the DV are contaminated and the contaminant can be removed mathematically through partialling.”
Control Variables in Leadership Research: A Qualitative and Quantitative Review (2017, methodology): “…2012), whether such variables are perceptual or objective (Atnic et al., 2012), and the complications associated with the inclusion of controls (Becker, 2005; Breaugh, 2006; Spector & Brannick, 2011), yet no study documents what is actually being controlled for across existing leadership areas.”
The Rise of Socially Responsible Investment Funds: The Paradoxical Role of the Financial Logic (2019, background): “Another potential concern arises from the possibility that the result is a mere statistical artifact (Becker, 2005; Carlson and Wu, 2011; Spector and Brannick, 2011) driven by the choice of control variables.”
Service employee burnout and engagement: the moderating role of power distance orientation (2016, background): “This relationship is statistically different 3 Control variables should be employed to rule out alternative explanations while testing a model under consideration; however, they must be carefully chosen based on their theoretical relevance and significant zeroorder correlations with the main constru”
Episodes of incivility between subordinates and supervisors: Examining the role of self‐control and time with an interaction‐record diary study. (2015, background): “According to Spector and Fox (2005), incivility is the mildest of these behaviors; examples of incivility include ignoring someone’s greeting and not crediting work performed by others (Pearson, Andersson, & Wegner, 2001).”
Unpacking the relationship between empowering leadership and service-oriented citizenship behaviors: a multilevel approach (2014, neutral): “, aggregated) level of customer orientation, and tenure for their effects on customer learning climate to prevent any possible confounding effects that may result from dealerlevel factors and to explain additional variance in learning climate above and beyond the effect of empowering leadership (Car”