Conducting interactive experiments online

By Antonio A. Arechar, Simon Gächter, Lucas Molleman. Experimental Economics, 2017

316 citations recorded by the source corpus.

Online labor markets provide new opportunities for behavioral research, but conducting economic experiments online raises important methodological challenges. This particularly holds for interactive designs. In this paper, we provide a methodological discussion of the similarities and differences between interactive experiments conducted in the laboratory and online. To this end, we conduct a repeated public goods experiment with and without punishment using samples from the laboratory and the online platform Amazon Mechanical Turk. We chose to replicate this experiment because it is long and logistically complex. It therefore provides a good case study for discussing the methodological and practical challenges of online interactive experimentation. We find that basic behavioral patterns of cooperation and punishment in the laboratory are replicable online. The most important challenge of online interactive experiments is participant dropout. We discuss measures for reducing dropout and show that, for our case study, dropouts are exogenous to the experiment. We conclude that data quality for interactive experiments via the Internet is adequate and reliable, making online interactive experimentation a potentially valuable complement to laboratory studies.

Subfields: Management Science and Operations Research, Safety Research, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/dd2daf5f10c096a2fc68be99ade7deada1a790a9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Safety Research

In Sociology and Political Science

Citations in the corpus

MTurk Research: Review and Recommendations (2020, methodology): “Sources used to derive recommendations: 1 Antin and Shaw (2012); 2 Arechar, Gächter, and Molleman (2018); 3 Barends and de Vries (2019); 4 Bederson and Quinn (2011); 5 Behrend, Sharek, Meade, and Wiebe (2011); 6 Bergvall-Kåreborn and Howcroft (2014); 7 Brawley and Pury (2016); 8 Buchanan and…”
Requiem for a Nudge: Framing effects in nudging honesty (2020, unknown): “Recent literature points to the robustness, generalizability, and reproducibility of laboratory ﬁndings in online environments ( Arechar et al., 2018; Coppock et al., 2018; Snowberg and Yariv, 2018 ).”
Demographics and Dynamics of Mechanical Turk Workers (2018, background): “[3] investigate the use of MTurk for interactive experiments in the field of economics.”
Testing the Waters: Behavior across Participant Pools (2018, background): “…experiments and non-incentivized games, as well as (incentivized) repeated public goods and prisoner’s dilemma games (Paolacci et al., 2010; Horton et al., 2011; Berinsky et al., 2012; Goodman et al., 2013; Arechar et al., 2018).8 Hauser et al. (2019) offer a survey of research on the topic.”