Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations
By Melissa G. Keith, Louis Tay, P. D. Harms. Frontiers in Psychology, 2017
295 citations recorded by the source corpus.
Amazon Mechanical Turk (MTurk) is becoming a prevalent source of quick and cost effective data for organizational research, but there are questions about the appropriateness of the platform for organizational research. To answer these questions, we conducted an integrative review based on 75 papers evaluating the MTurk platform and 250 MTurk samples used in organizational research. This integrative review provides four contributions: (1) we analyze the trends associated with the use of MTurk samples in organizational research; (2) we develop a systems perspective (recruitment system, selection system, and work management system) to synthesize and organize the key factors influencing data collected on MTurk that may affect generalizability and data quality; (3) within each factor, we also use available MTurk samples from the organizational literature to analyze key issues (e.g., sample characteristics, use of attention checks, payment); and (4) based on our review, we provide specific recommendations and a checklist for data reporting in order to improve data transparency and enable further research on this issue.
Subfields: Computer Science Applications, Information Systems
View Paper: https://www.semanticscholar.org/paper/b0017fb224a0e9f8a994a31730ba241c8676f41c
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Science Applications
In Information Systems
Citations in the corpus
- MTurk Research: Review and Recommendations (2020, background): “To address ethical concerns, it is useful to provide detailed information related to time commitment required and compensation provided (Gleibs, 2017; Keith, Tay, & Harms, 2017).”
- Data quality of platforms and panels for online behavioral research (2021, unknown): “…for data collection, there are a number of recommendations for enhancing response validity that should be considered, such as via the use of standardized reporting checklists (for comprehensive reviews of such recommendations see Cheung et al., 2017; DeSimone et al., 2015; Keith et al., 2017).”