Beyond the Turk: Alternative Platforms for Crowdsourcing Behavioral Research

By Eyal Péer, S. Samat, Laura Brandimarte, Alessandro Acquisti. 2017

2,619 citations recorded by the source corpus.

Subfields: Applied Psychology, Computer Science Applications, Safety Research

View Paper: https://www.semanticscholar.org/paper/ca9f3a04a1b6524105f51676abdd198b118a7d18

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Computer Science Applications

In Safety Research

Citations in the corpus

Towards psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation (2021, neutral): “We conducted a preregistered randomised controlled trial on Prolific Academic with three conditions (an active condition, a ‘passive’ Infographics condition, and a control condition), across three languages: English (using a national sample of the United Kingdom), French and German.4 The active (ino”
The Brief State Rumination Inventory (BSRI): Validation and Psychometric Evaluation (2018, background): “ac), which has been proved to provide high-quality data (Peer et al. 2017).”
Advantages and Disadvantages of Using Internet-Based Survey Methods in Aviation-Related Research (2017, background): “This may not be a problem in the short term, but eventually demand will catch up with supply, and the cost of conducting online research will increase (Peer, Brandimarte, Samat, & Acquisti, 2017).”
Recruiting Participants With Programming Skills: A Comparison of Four Crowdsourcing Platforms and a CS Student Mailing List (2022, background): “Other studies measure the quality of work in crowdsourcing platforms such as MTurk, Prolific, and CrowdFlower [44, 45].”
An Identity Perspective on Ethical Leadership to Explain Organizational Citizenship Behavior: The Interplay of Follower Moral Identity and Leader Group Prototypicality (2019, methodology): “…have claimed that platform workers spend much of their time participating in studies and thus might have gained experience with common experimental tasks and questionnaires (Peer et al. 2017), we have several reasons to assume that these concerns do not reduce the reliability of our findings.”
Customers’ Willingness to Disclose Personal Information throughout the Customer Purchase Journey in Retailing: The Role of Perceived Warmth (2020, methodology): “We collected the data through the Prolific Academic (ProA) online crowdsourcing platform given some recent evidence that – compared to other platforms – its participants produce higher-quality data (Peer et al. 2017).”
Task-Dependent Algorithm Aversion (2019, background): “Prolific Academic is a crowdsourcing website in which participants are less familiar with common experimental paradigms and more honest than participants on MTurk (Peer et al. 2017).”
Human Decision Making with Machine Assistance (2019, result): “The skews in education and ethnicity are consistent with the findings from prior studies on crowd-working platform demographics [41, 54, 57].”
Data quality of platforms and panels for online behavioral research (2021, neutral): “Indeed, previous research has suggested that sites where participants ’ frequency of usage is lower (e.g., Prolific) show higher data quality (Peer et al., 2017).”
Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics (2018, background): “Later studies on the demographics of MTurk workers reported similar results (e.g., Goodman and Paolacci, 2017; Berg, 2015; Peer et al., 2017; Pavlick et al., 2014; Naderi, 2018; Difallah et al., 2018).”
Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA (2023, background): “[19] observed that MTurk (the platform with the most experienced workers) had worse data quality than Prolific (one of two platforms with less experienced workers) but better data quality than CrowdFlower (the other platform with less experienced workers).”
The quality of data collected online: An investigation of careless responding in a crowdsourced sample (2020, methodology): “Participants may further have their deceptive tendencies exacerbated by the incentive-structure of crowdsourcing platforms (J. Chandler, Mueller and Paolacci, 2014; J. Chandler, Paolacci, Peer, Mueller and Ratliff, 2015; Kan and Drummey, 2018; Peer et al., 2017; Stewart et al., 2017).”