Improving Anchoring Vignettes Designing Surveys to Correct Interpersonal Incomparability

By Daniel Hopkins, Gary King. 2010

205 citations recorded by the source corpus.

We report the results of several randomized survey experiments designed to evaluate two intended improvements to anchoring vignettes, an increasingly common technique used to improve interpersonal comparability in survey research. This technique asks for respondent self-assessments followed by assessments of hypothetical people described in vignettes. Variation in assessments of the vignettes across respondents reveals interpersonal incomparability, allowing researchers to improve comparability by rescaling self-assessments relative to vignette responses. Our experiments show, first, that switching the question order so that self-assessments follow the vignettes primes respondents to define the response scale in a common way. In this case, priming is not a bias to avoid but a means of better communicating the question’s meaning. Second, we demonstrate that combining vignettes and self-assessments in a single direct comparison induces inconsistent and considerably less informative responses. Since similar combined strategies are widely employed for related purposes, our results suggest that anchoring vignettes could reduce measurement error in many applications where they are not currently used. Data for our experiments come from a national telephone survey and a separate online survey.

Subfields: Health, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/1eed150216f652c11f8895e64db3630cf7336958

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health

In Sociology and Political Science

Citations in the corpus

Cross-Sample Comparisons and External Validity (2014, methodology): “Indeed, political scientists commonly rely on YouGov and the GfK Group – services that maintain large national subject pools of adults (e.g. Banks and Valentino 2012; Brooks and Geer 2007; Hopkins and King 2010).”
Resilience isn't the same for all: Comparing subjective and objective approaches to resilience measurement (2018, background): “One way of addressing both of these challenges may through be the use of anchoring vignettes (Hopkins & King, 2010).”
The French Unhappiness Puzzle: The Cultural Dimension of Happiness (2014, methodology): “It is fair to mention an appealing recent survey-design technique based on “anchoring vignettes”, which is meant to correct for self-assessment biases (King et al. 2004, King and Wand, 2006, Beegle et al. 2009, Kapteyn et al. 2009, Angelini et al. 2009, Hopkins and King, 2010).”
Measuring 'subjective resilience': using peoples' perceptions to quantify household resilience (2015, unknown): “One option is the use of anchoring vignettes (Hopkins & King, 2010), which provides people with a hypothetical example (such as the characterisation of a person highly vulnerable to flooding) and asks them to provide a subjective rating.”
How Censorship in China Allows Government Criticism but Silences Collective Expression (2013, methodology): “We thus adapt to the Chinese language the methodology introduced in the English language by Hopkins and King (2010).”
Modeling Ordered Choices: A Primer (2010, unknown): “Three rich sources are King and Wand (2007), Hopkins and King (2008) and King (2008). For another example, KSV (2007, p. 465, fn 6) provide a URL for the vignettes used in their study. King (2009) lists many additional sources in a large number of fields within the social sciences.”