“My Questionnaire is Too Long!” The assessments of motivational-affective constructs with three-item and single-item measures

By K. Gogół, Martin Brunner, Thomas Goetz, Romain Martin, Sonja Ugen, Ulrich Keller, Antoine Fischbach, F. Preckel. 2014

412 citations recorded by the source corpus.

Abstract Because testing time in educational research is typically scarce, the use of long scales to assess motivational-affective constructs can be problematic. The goal of the present study was to scrutinize the psychometric properties of short scales (with three items) and single-item measures for two core motivational-affective constructs (i.e., academic anxiety and academic self-concept) by conducting systematic comparisons with corresponding long scales across school subjects and within different subject domains (i.e., mathematics, German, French). Statistical analyses were based on representative data from 3879 ninth-grade students. All short forms possessed satisfactory levels of reliability (range: .75–.89) and substantial correlations with the long scales (range: .88–.97); correlational patterns with educational student characteristics (e.g., achievement, school satisfaction, gender, academic track, and socioeconomic status) were comparable to those obtained with the corresponding long scales (all average differences in correlations below .07). The correlational patterns between all single-item measures and the external criteria were similar to those obtained with the corresponding long scales (all average differences in correlations below .08), yet the single-item measures demonstrated low to modest score reliabilities (estimated with the model-based omega coefficient; range: .22–.72) and correlations with full scales (range: .50–.88). When long scales are not applicable, short forms and perhaps even single-item measures may represent psychometrically sound alternatives for assessing academic anxiety and academic self-concept for educational research purposes.

Subfields: Experimental and Cognitive Psychology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/cfc3b9fd3196fd0c641906a9d80974f41f35b011

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Intraindividual relations between achievement goals and discrete achievement emotions: An experience sampling approach (2016, methodology): “The reliability and validity of single-item measures has been supported by findings from Wanous, Reichers, and Hudy (1997) and a recent analysis by Gogol et al. (2014) focusing on motivational and emotional constructs in education.”
Measuring Teachers’ enjoyment, anger, and anxiety: The Teacher Emotions Scales (TES) (2016, result): “Notably, trait-based scales can easily be adapted for state assessments (Gogol et al., 2014), and we suggest that the TES can be adapted in a similar fashion.”
Teachers’ emotions and emotional authenticity: do they matter to students’ emotional responses in the classroom? (2020, methodology): “, 2015) and single items for emotional self-report showed adequate reliability (Gogol et al., 2014).”
The role of students' motivation and participation in predicting performance in a MOOC (2016, background): “More recently, Gogol et al. (2014) investigated the reliability and validity of three-item and single-item measures in relation to scores on full scales used to measure academic anxiety and academic self-concept in secondary students.”
Beyond cold technology: A systematic review and meta-analysis on emotions in technology-based learning environments (2018, positive): “%) of multi-item emotion measures implemented, sample-specific reliabilities were not reported; additionally, single-item assessment, which may differ considerably from multi-item approaches in terms of lower measurement accuracy (e.g., Gogol et al., 2014), was also prevalent.”
Consistency, longitudinal stability, and predictions of elementary school students' task interest, success expectancy, and performance in mathematics (2018, methodology): “As both single-item and post-task measures have been successfully used in previous studies (Ainley & Patrick, 2006; Durik & Harackiewicz, 2007; Gogol et al., 2014; Silvia, 2003), this should not be taken as a serious threat to the validity of the findings.”
The Relations Among School Climate, Instructional Quality, and Achievement Motivation in Mathematics (2016, positive): “Although this decision limited the overall number of indicators of self-concept, existing research has shown that a three-item scale still provides a reliable and valid measure of students ’ self-concept (Gogol et al. 2014).”
The influence of the high school classroom environment on learning as mediated by student engagement (2017, methodology): “The use of single-item measures is supported by an increasing number of studies demonstrating their utility and acceptable psychometric properties (Gogol et al., 2014; Grice, Melissa, & Badzinski, 2011), but remains debatable.”
Interest-based text preference moderates the effect of text difficulty on engagement and learning (2015, background): “…that single-item measures can have acceptable reliability and validity, particularly if the purpose of the measure is clear, the item accurately represents the underlying construct, and the assessment i contextually-grounded and occurs in the moment (Ainley et al., 2005; Gogel et al., 2014).”