How to measure metacognition

By Stephen M. Fleming, H. Lau. Frontiers in Human Neuroscience, 2014

1,105 citations recorded by the source corpus.

The ability to recognize one's own successful cognitive processing, in e.g., perceptual or memory tasks, is often referred to as metacognition. How should we quantitatively measure such ability? Here we focus on a class of measures that assess the correspondence between trial-by-trial accuracy and one's own confidence. In general, for healthy subjects endowed with metacognitive sensitivity, when one is confident, one is more likely to be correct. Thus, the degree of association between accuracy and confidence can be taken as a quantitative measure of metacognition. However, many studies use a statistical correlation coefficient (e.g., Pearson's r) or its variant to assess this degree of association, and such measures are susceptible to undesirable influences from factors such as response biases. Here we review other measures based on signal detection theory and receiver operating characteristics (ROC) analysis that are “bias free,” and relate these quantities to the calibration and discrimination measures developed in the probability estimation literature. We go on to distinguish between the related concepts of metacognitive bias (a difference in subjective confidence despite basic task performance remaining constant), metacognitive sensitivity (how good one is at distinguishing between one's own correct and incorrect judgments) and metacognitive efficiency (a subject's level of metacognitive sensitivity given a certain level of task performance). Finally, we discuss how these three concepts pose interesting questions for the study of metacognition and conscious awareness.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/a2e0e3ba3831b3acaed5de5899241ebbccaa1016

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation (2017, background): “Metacognitive accuracy refers to the relationship between self-evaluation and performance, and comprises two components: sensitivity and bias 3 (Fleming & Lau, 2014).”
Toward Untangling Creative Self-Beliefs (2017, background): “…between a person’s level of confidence in performing a task and that person’s actual level of performance—yielding four possibilities (see also Fleming & Lau, 2014d): high confidence, high performance (accurate high alignment); high confidence, low performance (inaccurate overestimation);…”
The Metacognitive Demands and Opportunities of Generative AI (2023, background): “…arising from metacognitive knowledge and experiences—e.g., the difference between the ability to be self-aware about one’s memory (monitoring) and the information conveyed by a feeling of familiarity (an experience). is subjectively experienced rather than consciously known, and because…”
The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis (2020, background): “Fleming and Lau (2014) make the case that simple correlations may be insufficient for understanding the degree of correspondence between accuracy and meta-cognitive processes, due to the presence of response biases (i.e., the general propensity toward over- or underconfidence of the individual).”