Investigating an alternate form of the cognitive reflection test

By K. S. Thomson, Daniel M. Oppenheimer. Judgment and Decision Making, 2016

476 citations recorded by the source corpus.

Much research in cognitive psychology has focused on the tendency to conserve limited cognitive resources. The CRT is the predominant measure of such miserly information processing, and also predicts a number of frequently studied decision-making traits (such as belief bias and need for cognition). However, many subjects from common subject populations have already been exposed to the questions, which might add considerable noise to data. Moreover, the CRT has been shown to be confounded with numeracy. To increase the pool of available questions and to try to address numeracy confounds, we developed and tested the CRT-2. CRT-2 questions appear to rely less on numeracy than the original CRT but appear to measure closely related constructs in other respects. Crucially, substantially fewer subjects from Amazon’s Mechanical Turk have been previously exposed to CRT-2 questions. Though our primary purpose was investigating the CRT-2, we also found that belief bias questions appear suitable as an additional source of new items. Implications and remaining measurement challenges are discussed.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, General Decision Sciences

View Paper: https://www.semanticscholar.org/paper/37967601394bd4e14629be5e1d95a94eb949338f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In General Decision Sciences

Citations in the corpus

Analytic-thinking predicts hoax beliefs and helping behaviors in response to the COVID-19 pandemic (2020, methodology): “We used a seven-item CRT: The original three-item CRT (Frederick, 2005) was included along with the four items from the non-numeric version of the CRT developed by Thomson and Oppenheimer (2016).”
On the belief that beliefs should change according to evidence: Implications for conspiratorial, moral, paranormal, political, religious, and science beliefs (2019, methodology): “…reli- gious belief questionnaire to be consistent with the paranor- mal/political/science questionnaires (i.e., a 7-point scale); 4) we added 3 CRT items from Thomson and Oppenheimer (2016) that are relatively easier, based on past research (see https://osf.io/xqzse/ for full materials); and 5) we…”
Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention (2020, methodology): “We gave participants a six-item Cognitive Reflection Test (CRT; Frederick, 2005) that consisted of a reworded version of the original three-item test and three items from a nonnumeric version (we excluded the “hole” item; Thomson & Oppenheimer, 2016).”
Demographics and Dynamics of Mechanical Turk Workers (2018, background): “Sociology, psychology, behavioral, economics, political science researchers actively recruit study participants on various crowdsourcing platforms [2, 3, 5, 23, 32, 33, 35].”
Political Ideology Predicts Perceptions of the Threat of COVID-19 (and Susceptibility to Fake News About It) (2020, background): “The CRT contained nine items (a ¼ .735) taken from several sources (Baron et al., 2015; Oldrati et al., 2016; Primi et al., 2016; Thomson & Oppenheimer, 2016; Toplak et al., 2014).”
Explanations in the wild. (2021, unknown): “Cognitive reflection: We combined 3 Cognitive Reflection Test (CRT) items from Shenhav et al. (2012) and 4 items from Thomson and Oppenheimer (2016).”