An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use

By Michael D. Buhrmester, Sanaz Talaifar, Samuel D. Gosling. Perspectives on Psychological Science, 2018

645 citations recorded by the source corpus.

Subfields: Computer Science Applications, Ecological Modeling, Health

View Paper: https://www.semanticscholar.org/paper/1d0588ab31f2e29c6029cfa0762bad255c30d94a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Ecological Modeling

In Health

Citations in the corpus

On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices (2021, background): “Surprisingly, guidelines for reporting crowdsourcing experiments have received little attention even though underreporting is a serious concern in science, and crowdsourcing is no exception [7].”
Data quality of platforms and panels for online behavioral research (2021, neutral): “In the past decade, there has been a proliferation of online platforms for research (e.g., Buhrmester et al., 2018; Goodman & Paolacci, 2017).”
The Influence of Consumer–Brand Relationship on the Personalized Advertising Privacy Calculus in Social Media (2021, background): “As noted above, MTurk samples have been shown to have greater demographic diversity and representativeness of the US population compared with standard Internet and convenience samples (Buhrmester et al., 2018).”
MTurk Research: Review and Recommendations (2020, methodology): “In addition, it is useful to include procedures designed to capture an MTurkers’ IP address and use features that prevent the same MTurker from completing the study more than once (i.e., avoiding “ballot box stuffing”; Buhrmester et al., 2018; Chandler & Paolacci, 2017).”
The Expertise Involved in Deciding which HITs are Worth Doing on Amazon Mechanical Turk (2021, background): “Studies have found that these imbalances in fairness directly affect worker satisfaction of the platform [4, 10] and can negatively impact the quality of work that academic studies depend on [6, 25, 40].”
Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA (2023, background): “As online data collection for human subjects through platforms like Amazon’s Mechanical Turk (MTurk) becomes increasingly common [1], so too have concerns over the quality of these data.”
How serious is the ‘carelessness’ problem on Mechanical Turk? (2019, background): “Since 2005, myths about online participants as social misfits were largely dispelled, and companies such as Qualtrics offered methodological tools to facilitate the collection of research data on the Internet (Buhrmester et al., 2018).”
Basic Psychological Need Satisfaction, Autonomous Motivation, and Meaningful Work: A Self-Determination Theory Perspective (2021, methodology): “It allows for recruitment of more representative samples than those obtained through traditional methods (Buhrmester et al., 2018).”
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results (2019, methodology): “Recent years have seen an exponential increase in psychological studies being conducted on MTurk (Buhrmester, Talaifar, & Gosling, 2018), likely because MTurk is an efficient method for collecting large amounts of data (Buhrmester, Kwang, & Gosling, 2011; Buhrmester et al., 2018; Stewart et al.,…”