An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets

By Jakob Rogstadius, Vassilis Kostakos, Aniket Kittur, Boris Smus, Jim Laredo, Maja Vuković. International Conference on Web and Social Media, 2011

358 citations recorded by the source corpus.

Crowdsourced labor markets represent a powerful new paradigm for accomplishing work. Understanding the motivating factors that lead to high quality work could have significant benefits. However, researchers have so far found that motivating factors such as increased monetary reward generally increase workers’ willingness to accept a task or the speed at which a task is completed, but do not improve the quality of the work. We hypothesize that factors that increase the intrinsic motivation of a task – such as framing a task as helping others – may succeed in improving output quality where extrinsic motivators such as increased pay do not. In this paper we present an experiment testing this hypothesis along with a novel experimental design that enables controlled experimentation with intrinsic and extrinsic motivators in Amazon’s Mechanical Turk, a popular crowdsourcing task market. Results suggest that intrinsic motivation can indeed improve the quality of workers’ output, confirming our hypothesis. Furthermore, we find a synergistic interaction between intrinsic and extrinsic motivators that runs contrary to previous literature suggesting “crowding out” effects. Our results have significant practical and theoretical implications for crowd work.

Subfields: Computer Science Applications, Safety Research

View Paper: https://www.semanticscholar.org/paper/2e4d6082439de78101366a760f9d6d2c51d86fdb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

Citations in the corpus

Data Collection for Dialogue System: A Startup Perspective (2018, background): “In terms of the variations we consider, the closest work is Rogstadius et al. (2011), who also considered how task framing can impact behavior.”
On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices (2021, background): “Existing guidelines and best practices focus on how to run experiments in crowdsourcing environments successfully, proposing strategies to overcome common pitfalls found in crowdsourcing platforms [9, 38, 46, 64].”
Monopsony in Online Labor Markets (2020, background): “5We examined the estimates in the following papers: Berinsky et al. (2012), Buhrmester et al. (2011), Crump et al. (2013), Doerrenberg et al. (2016), Heer and Bostock (2010), Horton and Chilton (2010), Marge et al. (2010), Mason and Watts (2009), Rogstadius et al. (2011), Sorokin and Forsyth (2008)”
Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations (2017, background): “There are beneﬁts to providing higher compensation as it is related to greater speed and quantity of data collection; likely because workers can sort HITs by level of payment (Mason and Watts, 2010; Faridani et al., 2011; Rogstadius et al., 2011 ).”
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets (2017, background): “Prevention techniques include screening for crowdworkers capable of different tasks [18, 27], pre-training crowdworkers [19], maintaining quality while controlling for cost via dynamic task allocation [52, 8], or designing interfaces or payment structures to motivate good work [43, 47, 22].”