TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences

By Leib Litman, Jonathan Robinson, Tzvi Abberbock. Behavior Research Methods, 2016

1,783 citations recorded by the source corpus.

In recent years, Mechanical Turk (MTurk) has revolutionized social science by providing a way to collect behavioral data with unprecedented speed and efficiency. However, MTurk was not intended to be a research tool, and many common research tasks are difficult and time-consuming to implement as a result. TurkPrime was designed as a research platform that integrates with MTurk and supports tasks that are common to the social and behavioral sciences. Like MTurk, TurkPrime is an Internet-based platform that runs on any browser and does not require any downloads or installation. Tasks that can be implemented with TurkPrime include: excluding participants on the basis of previous participation, longitudinal studies, making changes to a study while it is running, automating the approval process, increasing the speed of data collection, sending bulk e-mails and bonuses, enhancing communication with participants, monitoring dropout and engagement rates, providing enhanced sampling options, and many others. This article describes how TurkPrime saves time and resources, improves data quality, and allows researchers to design and implement studies that were previously very difficult or impossible to carry out on MTurk. TurkPrime is designed as a research tool whose aim is to improve the quality of the crowdsourcing data collection process. Various features have been and continue to be implemented on the basis of feedback from the research community. TurkPrime is a free research platform.

Subfields: Applied Psychology, Computer Science Applications, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/2f35b305b6c56046b631b0cdb6d2f08e4ee577a7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Computer Science Applications

In Human-Computer Interaction

Citations in the corpus

Explanations in the wild. (2021, positive): “We used CloudResearch/TurkPrime to manage participation as this platform tracks data quality in MTurk workers (Litman et al., 2017).”
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations (2017, methodology): “Consistent with Litman et al. (2016), we recommend using a dynamic code that is unique for each worker to avoid the potential for the code being shared on message boards.”
MTurk Character Misrepresentation: Assessment and Solutions (2017, background): “There are emerging enterprises, such as TurkPrime (Litman, Robinson, and Abberbock 2016) and Prolific Academic (ProA), that allow screening for a lower fee.”
The influence of demographics and personality on COVID-19 coping in young adults (2020, neutral): “Using CloudResearch functions (Litman et al., 2017), the study was limited to young adults (24–35 years of age).”
Automated rationale generation: a technique for explainable AI and its effects on human perceptions (2019, methodology): “To gather the training set of game state annotations, we deployed our data collection pipeline on Turk Prime [28].”
Crowdsourcing Consumer Research (2017, background): “Requiring enrollment prevents previewing of a study and raises the time costs required by participants to return the task to MTurk (Litman et al. 2016; Peer et al. 2012).”
Political Ideology Predicts Perceptions of the Threat of COVID-19 (and Susceptibility to Fake News About It) (2020, methodology): “We used CloudResearch to conduct this study (Litman et al., 2016), and participants completed it on March 8, 2020.”
An Identity Perspective on Ethical Leadership to Explain Organizational Citizenship Behavior: The Interplay of Follower Moral Identity and Leader Group Prototypicality (2019, methodology): “To establish the experimental paradigm, we recruited 106 participants from the USA (mean age = 36.33 years, SD = 9.88; 34.9 females) using TurkPrime (Litman et al. 2017) for a financial compensation of USD 0.80.”