An Analysis of Data Quality: Professional Panels, Student Subject Pools, and Amazon's Mechanical Turk

By J. Kees, Christopher Berry, Scot Burton, Kim Bartel Sheehan. 2017

842 citations recorded by the source corpus.

Subfields: Marketing, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/be78d6151c5724854a034aad204d20902c5d013f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Marketing

In Sociology and Political Science

Citations in the corpus

Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA (2023, background): “Other researchers observed that MTurk produces data of comparable quality to student samples and superior quality data to other forms of professional platforms like Qualtrics and Lightspeed [14].”
Amazon's Mechanical Turk: A Comment (2017, positive): “Kees et al. (2017) provide evidence from a comparative study that supports the use of Mechanical Turk (MTurk) data in advertising research when compared to student samples and online panel data.”
Initial Challenges of Caregiving During COVID-19: Caregiver Burden, Mental Health, and the Parent–Child Relationship (2020, background): “Online survey data management guidelines suggest removing cases of substandard completion or inattentiveness, determined by abnormally quick response times [64, 65].”
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results (2019, background): “…or superior in quality to data collected in the lab, from professional online panels, and using marketing research companies (Behrend, Sharek, Meade, & Wiebe, 2011; Buhrme-ster et al., 2011; Casler, Bickel, & Hackett, 2013; Kees, Berry, Burton, & Sheehan, 2017; Paolacci & Chandler, 2014).”
Toward a Relational Theory of Employee Engagement: Understanding Authenticity, Transparency, and Employee Behaviors (2020, background): “Although Mturk participants are self-selected rather than randomly recruited, data collected using Mturk can be of a high quality (Kees et al., 2017; Ross et al., 2010).”
Demographics and Dynamics of Mechanical Turk Workers (2018, result): “Many studies show that the data collected using MTurk workers is comparable in quality to the data collected using undergraduate students or professional/commercial panels [19].”
Crowdsourcing research: Data collection with Amazon’s Mechanical Turk (2018, background): “Keep in mind, though, that Kees et al. (2017) found that MTurk workers completed a study in about two-thirds of the time it took for student and Qualtrics samples and in about half of the time for the Lightspeed sample.”
Individuals’ intentions to engage in last chance tourism: applying the value-belief-norm model (2020, methodology): “As Lemelin et al. (2010) advances, LCT can be thought of as individuals’ desire and pursuit of visiting landscapes and viewing key species whose existence is threatened or in jeopardy of disappearing.”