Crowdsourcing research: Data collection with Amazon’s Mechanical Turk

By Kim Bartel Sheehan. 2018

379 citations recorded by the source corpus.

Subfields: Computer Science Applications, Information Systems, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/4aa9cae6872c26b7dfd870d29b8de5d663a264e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Sociology and Political Science

Citations in the corpus

An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results (2019, background): “…or superior in quality to data collected in the lab, from professional online panels, and using marketing research companies (Behrend, Sharek, Meade, & Wiebe, 2011; Buhrme-ster et al., 2011; Casler, Bickel, & Hackett, 2013; Kees, Berry, Burton, & Sheehan, 2017; Paolacci & Chandler, 2014).”
Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions (2018, methodology): “While recruiting respondents using this method is increasingly done in communication research (Sheehan, 2018), theseworkersmay not be representative of typical users of consumer websites.”
The Expertise Involved in Deciding which HITs are Worth Doing on Amazon Mechanical Turk (2021, methodology): “While some Turkers have been found to be skeptical of TurkOpticon ratings [48], many Turkers regularly use TurkOpticon before deciding to accept and complete a HIT.”
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021, methodology): “We provided them with a definition of hate speech (Twitter, 2021) and examples of explicit, implicit, and nonhateful content (See Appendix A), and required them to pass a short five-question qualification check for understanding with a score of at least 90% in accordance with crowdsourcing standards”
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems (2022, neutral): “Following ethical crowdsourcing guidelines out-lined in Sheehan (2018), we trained Amazon Mechanical Turk (MTurk) workers to complete all annotations described in this study.”
Keeping up with influencers: exploring the impact of social presence and parasocial interactions on Instagram (2021, methodology): “…form and were directed automatically to a preliminary questionnaire, followed by the stimulus material and a final set of questions. us consumers were recruited through MTurk. studies have shown that MTurk data are as reliable and valid as data gathered through traditional methods (sheehan 2018).”
Initial Challenges of Caregiving During COVID-19: Caregiver Burden, Mental Health, and the Parent–Child Relationship (2020, background): “Online survey data management guidelines suggest removing cases of substandard completion or inattentiveness, determined by abnormally quick response times [64, 65].”