Internal Consistency, Retest Reliability, and their Implications For Personality Scale Validity

By R. McCrae, J. E. Kurtz, S. Yamagata, Antonio Terracciano. Personality and Social Psychology Review, 2010

819 citations recorded by the source corpus.

The authors examined data (N = 34,108) on the differential reliability and validity of facet scales from the NEO Inventories. They evaluated the extent to which (a) psychometric properties of facet scales are generalizable across ages, cultures, and methods of measurement, and, (b) validity criteria are associated with different forms of reliability. Composite estimates of facet scale stability, heritability, and cross-observer validity were broadly generalizable. Two estimates of retest reliability were independent predictors of the three validity criteria; none of three estimates of internal consistency was. Available evidence suggests the same pattern of results for other personality inventories. Internal consistency of scales can be useful as a check on data quality but appears to be of limited utility for evaluating the potential validity of developed scales, and it should not be used as a substitute for retest reliability. Further research on the nature and determinants of retest reliability is needed.

Subfields: Clinical Psychology, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/4a02987b992064ff4d81b6512ac57bac5523c58b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Clinical Psychology

In Experimental and Cognitive Psychology

Citations in the corpus

A meta-analysis of dependability coefficients (test–retest reliabilities) for measures of the Big Five (2014, background): “Therefore, this study answers the repeated call for a greater emphasis of dependability in personality research (McCrae et al., 2011; Schmidt et al. 2003; Watson, 2004) and presents a comprehensive meta-analysis of dependability coefficients for measures of the Big Five that also acknowledges the ch”
Psychometric Properties of a Revised Version of the Ten Item Personality Inventory (2015, background): “However, as suggested by Gosling et al. (2003) and McCrae et al. (2011), test-r test reliability coefficients may provide better estimates of reliability, especially for short scales, and Study 4 provided evidence that for the I-TIPI-R such values were adequate.”
Who Is Impacted? Personality Predicts Individual Differences in Psychological Consequences of the COVID-19 Pandemic in Germany (2020, background): “…internal consistencies are usually lower than retest reliabilities (McCrae et al., 2011) and probably less relevant to the validity of a scale (McCrae et al., 2011, McCrae & Mõttus, 2019) Fourth, there is an ongoing debate of what partial effects mean and whether they should be interpreted…”
Predicting Counterproductive Work Behavior from Guilt Proneness (2012, background): “For further discussion of the limitations of internal consistency as an indicator of reliability, see McCrae et al. 2011; Schmitt 1996.”