Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research

By Gary King, Christopher J. L. Murray, J. Salomon, A. Tandon. American Political Science Review, 2003

1,129 citations recorded by the source corpus.

We address two long-standing survey research problems: measuring complicated concepts, such as political freedom and efficacy, that researchers define best with reference to examples; and what to do when respondents interpret identical questions in different ways. Scholars have long addressed these problems with approaches to reduce incomparability, such as writing more concrete questions—with uneven success. Our alternative is to measure directly response category incomparability and to correct for it. We measure incomparability via respondents' assessments, on the same scale as the self-assessments to be corrected, of hypothetical individuals described in short vignettes. Because the actual (but not necessarily reported) levels of the vignettes are invariant over respondents, variability in vignette answers reveals incomparability. Our corrections require either simple recodes or a statistical model designed to save survey administration costs. With analysis, simulations, and cross-national surveys, we show how response incomparability can drastically mislead survey researchers and how our approach can alleviate this problem.

Subfields: Political Science and International Relations, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/a47dff54e7bc4f49d26092aef46fa9a386199cb6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Political Science and International Relations

In Sociology and Political Science

Citations in the corpus

Improving the Measurement of Food Security (2012, positive): “12 One proposed means of improving subjective data is to use “anchoring vignettes” to elicit common reference frames (King et al. 2004).”
Happiness Economics from 35,000 Feet (2012, positive): “Anchoring vignettes Data sets in which anchoring vignette items are included alongside an SWB self-assessment permit two further approaches to correcting for differing reporting functions (R) across individuals, given two key assumptions (King et al., 2004).”
Improving Anchoring Vignettes Designing Surveys to Correct Interpersonal Incomparability (2010, methodology): “…and Szwarcwald 2005).2 Improving the statistical methods for analyzing anchoring vignette data has attracted considerable scholarly attention (King et al. 2004; Javaras and Ripley 2007; King and Wand 2007; Soest et al. 2007; Wand 2007; Gupta, Kristensen, and Pozzoli 2008), but less work has…”
True health vs response styles: exploring cross-country differences in self-reported health. (2007, background): “The generalised ordered probit model makes threshold parameters dependent on covariates [21,22].”
Using Bayesian Aldrich-McKelvey Scaling to Study Citizens' Ideological Preferences and Perceptions (2015, background): “Scholars have long recognized the prevalence of interpersonal incomparability or DIF—a problem that arises when respondents interpret and answer survey items in different ways— in public opinion data (Brady, 1985; King et al., 2004; Bakker, Jolly, Polk and Poole, 2014).”
The Sad Truth about Happiness Scales (2019, background): “4 See Lehmann (1955).”
The Political Economy of Women's Support for Fundamentalist Islam (2008, neutral): “On this issue, see King et al. (2004).”
Do Danes and Italians Rate Life Satisfaction in the Same Way? Using Vignettes to Correct for Individual‐Specific Scale Biases (2014, neutral): “Anchoring vignettes were first introduced by King et al. (2004) for analyzing ordinal survey re-”
The French Unhappiness Puzzle: The Cultural Dimension of Happiness (2014, background): “Finally, international comparisons of happiness are necessarily related to the large literature that focuses on biases and equivalence between constructs, measures and scales (Van de Vijver 1998, King et al. 2003)3.”