Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets

By Dana Chandler, Adam Kapelner. arXiv.org, 2012

432 citations recorded by the source corpus.

We conduct the first natural field experiment to explore the relationship between the “meaningfulness” of a task and worker effort. We employed about 2500 workers from Amazon's Mechanical Turk (MTurk), an online labor market, to label medical images. Although given an identical task, we experimentally manipulated how the task was framed. Subjects in the meaningful treatment were told that they were labeling tumor cells in order to assist medical researchers, subjects in the zero-context condition (the control group) were not told the purpose of the task, and, in stark contrast, subjects in the shredded treatment were not given context and were additionally told that their work would be discarded. We found that when a task was framed more meaningfully, workers were more likely to participate. We also found that the meaningful treatment increased the quantity of output (with an insignificant change in quality) while the shredded treatment decreased the quality of output (with no change in quantity). We believe these results will generalize to other short-term labor markets. Our study also discusses MTurk as an exciting platform for running natural field experiments in economics.

Subfields: Computer Science Applications, Safety Research, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/296a1d392f96e2f4094a9324156a10e8e0e3bb27

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

In Sociology and Political Science

Citations in the corpus

Shepherding the crowd yields better work (2012, background): “Moreover, providing context about the rationale for a task can be a powerful motivator [5].”
What makes work meaningful and why economists should care about it (2020, background): “task meaning) increases effort and productivity (Ariely et al., 2008; Bäker and Mechtel, 2018; Chadi et al., 2016; Chandler and Kapelner, 2013; Grant, 2008; Kosfeld et al., 2017) For example, Ariely et al.”
An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets (2011, methodology): “Finally, the task used in Chandler & Kapelner (2010) resulted in very high performance levels, suggesting a possible ceiling effect on the influence of intrinsic motivation.”
The online laboratory: conducting experiments in a real labor market (2010, neutral): “In addition to a $.20 “show-up fee,” subjects were further informed of the following payoﬀ structure (in units of cents):”
Making Experience Count: The Role of Reflection in Individual Learning (2016, methodology): “This task was adapted from the paradigm used by Chandler and Kapelner (2013).”
Crowdsourcing Consumer Research (2017, methodology): “Finally, many workers participate in MTurk for reasons additional to earning money, and are often both intrinsically and extrin-sically motivated (Chandler and Kapelner 2013; Horton, Rand, and Zeckhauser 2011; Paolacci et al. 2010).”
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Accuracy (2016, background): “Although crowdsourcing has shown to have the potential to be intrinsically motivating [54], the microtask paradigm found in the majority of crowdsourcing tasks favors a structure that is efficient [15,26] for workers rather than being interesting for them [7].”
The Future of Work in the ‘Sharing Economy’. Market Efficiency and Equitable Opportunities or Unfair Precarisation? (2016, neutral): “This is interpreted as 36 the importance of telling workers the meaning of the work (Chandler & Kapelner, 2013).”
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality (2016, background): “Previous work directly targets the monotony of crowdsourcing, showing that by framing the task as more meaningful to workers (for example as a charitable cause), one obtains higher quality results [7].”
Inside the Turk (2014, background): “Thanking workers and explaining to them the meaning of the task they will complete can stimulate better work (D. Chandler & Kapelner, 2013), as does framing a task as requested by a nonprofit organization (Rogstadius et al., 2011).”
Running experiments on Amazon Mechanical Turk (2010, result): “In fact, 69.6% of the U.S.-based workers reported that they consider Mechani-JudgmentandDecisionMaking cal Turk is a fruitful way to spend free time (e.g., instead of watching TV), a ﬁnding which is consistent with previous results (Chandler & Kapelner, 2010; Horton, Rand & Zeckhauser, 2010).”