Captions, consistency, creativity, and the consensual assessment technique: New evidence of reliability

By James C. Kaufman, Joohyung Lee, John Baer, Soonmook Lee. 2007

144 citations recorded by the source corpus.

Abstract The consensual assessment technique (CAT) is a measurement tool for creativity research in which appropriate experts evaluate creative products [Amabile, T. M. (1996). Creativity in context: Update to the social psychology of creativity . Boulder, CO: Westview]. However, the CAT is hampered by the time-consuming nature of the products (asking participants to write stories or draw pictures) and the ratings (getting appropriate experts). This study examined the reliability of ratings of sentence captions. Specifically, four raters evaluated 12 captions written by 81 undergraduates. The purpose of the study was to see whether the CAT could provide reliable ratings of captions across raters and across multiple captions and, if so, how many such captions would be required to generate reliable scores, and how many judges would be needed? Using generalizability theory, we found that captions appear to be a useful way of measuring creativity with a reasonable level of reliability in the frame of CAT.

Subfields: Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/5d1223894bafba734edaef6dc3ef74766afa99b6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

Citations in the corpus

A snapshot of creativity: Evaluating a quick and simple method for assessing divergent thinking (2009, background): “Recent work has loosened the classic CAT to include diverse kinds of products, raters, and judgments (Baer et al., 2004, Kaufman et al., 2005 and Kaufman et al., 2007).”
Making creative metaphors: The importance of fluid intelligence for creative thought (2012, background): “past reviews and Kim's (2005) meta-analysis conclude otherwise, so an executive interpretation of creative thought is understandably controversial.”
Beyond New and Appropriate: Who Decides What Is Creative? (2012, background): “If one asks participants to write creative captions for photographs or titles for stories, who would have the appropriate expertise to judge those captions or titles? Editors, perhaps? Or might any reasonably well-read person be an appropriate expert in such instances? Kaufman, Lee, Baer, and Lee ( ”
Approaches to Measuring Creativity: A Systematic Literature Review (2017, methodology): “However, some situational variables seem to influence this reliability, such as the number of tasks being judged (Lee, Lee, & Youn, 2005; Kaufman et al., 2007) and the performance domain (Baer, 1991; Baer et al.”