Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study

By B. Gillen, Erik Snowberg, Leeat Yariv. Journal of Political Economy, 2019

334 citations recorded by the source corpus.

Measurement error is ubiquitous in experimental work. It leads to imperfect statistical controls, attenuated estimated effects of elicited behaviors, and biased correlations between characteristics. We develop statistical techniques for handling experimental measurement error. These techniques are applied to data from the Caltech Cohort Study, which conducts repeated incentivized surveys of the Caltech student body. We replicate three classic experiments, demonstrating that results change substantially when measurement error is accounted for. Collectively, these results show that failing to properly account for measurement error may cause a field-wide bias leading scholars to identify “new” phenomena.

Subfields: General Decision Sciences, Management Science and Operations Research, Safety Research

View Paper: https://www.semanticscholar.org/paper/ec144179171a6fffe09c7d9334ac2b58d9a62629

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Decision Sciences

In Management Science and Operations Research

In Safety Research

Citations in the corpus

Digging into the Pocketbook: Evidence on Economic Voting from Income Registry Data Matched to a Voter Survey (2017, background): “See Online Appendix A for detailed information on the register data.”
Online Appendix (2020, positive): “If the concerns raised by Gillen et al. (2019) are valid in our data, they could steer us towards incorrect inferences.”
The psychometric and empirical properties of measures of risk preferences (2017, neutral): “The research that does address or touch upon these questions suggests the measures are subject to significant measurement error (Barsky et al. 1997; Gillen et al. 2015; Harrison et al. 2005; Sahm 2012; Kimball et al. 2008; Lönnqvist et al. 2014) and have only limited predictive validity.”
Do Women Give Up Competing More Easily? Evidence from the Lab and the Dutch Math Olympiad (2016, background): “The size of the markers is proportional to the amount of observations in that particular bin ( score and gender . bandwidth of 4 and for the quadratic approach, we start with a bandwidth of 7.”
Designing Information Provision Experiments (2020, background): “How can researchers mitigate the extent of measurement error? Gillen et al. (2019) propose an IV approach, which leverages multiple measurements to deal with classical measurement error.”
The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences (2016, background): “Measurement error in the survey module can attenuate explanatory power for incentivize choices in experiments, or other outcome variables, as well as make the module items imperfect statistical controls (for discussions see, e.g., Spearman 1904, Gillen et al. 2019).”
Testing the Waters: Behavior across Participant Pools (2018, methodology): “These mean differences do not tend to lead to disagreement in the signs (23)For further details, see Gillen et al. (2018). (24)These scores are derived from the differences between mean latencies across the two combined classification stages in each of the IATs, see Greenwald et al. (2003). The gend”