Financial incentives and the "performance of crowds"

By Winter A. Mason, Duncan J. Watts. SKDD, 2010

995 citations recorded by the source corpus.

Subfields: Computer Science Applications, Safety Research

View Paper: https://www.semanticscholar.org/paper/966d7d8ff1bfb41d89bdd49dcc059727845e5794

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

Citations in the corpus

Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations (2017, background): “There are beneﬁts to providing higher compensation as it is related to greater speed and quantity of data collection; likely because workers can sort HITs by level of payment (Mason and Watts, 2010; Faridani et al., 2011; Rogstadius et al., 2011 ).”
Crowdsourced Data Management: Industry and Academic Perspectives (2015, background): “These include studies on how honest workers are [174], what kinds of tasks workers enjoy [97, 177], whether crowdsourcing marketplaces are a good testbed for user studies [115, 117], how pricing impacts worker behavior [134, 96], and how often spam or bias occurs [100, 137].”
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use (2018, background): “…how much impact our article had on the resulting explosion of research using and evaluating MTurk, given that a number of other similar articles came out at about the same time (e.g., Berinsky, Huber, & Lenz, 2012; Mason & Watts, 2009; Paolacci et al., 2010), most of which are also highly cited.”
Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms (2015, background): “Called a “market for lemons” (Ipeirotis, 2010d) and “digital sweatshop” (Cushing, 2012), AMT offers little support or guidance on task pricing (Mason & Watts, 2009; Faridani, Hartmann, & Ipeirotis, 2011; Singer & Mittal, 2011).”
Truthful incentives in crowdsourcing tasks using regret minimization mechanisms (2013, background): “[28] [29] study other, non-monetary incentives that could improve the quality of workers' performance.”
Corpus Annotation through Crowdsourcing: Towards Best Practice Guidelines (2014, result): “While some initial reports found that high rewards attracted noise that was detrimental to quality (Mason and Watts, 2010), more recent research was unable to repeat this ﬁnding, reporting that increased reward only affected the number of workers attracted to a job (and thus sped up its overall…”
Collaborative learning in networks (2011, background): “Although virtual laboratories suffer from certain disadvantages relative to their physical counterparts, several recent studies have shown that many of these disadvantages can be overcome (15 – 17).”
Evaluation on crowdsourcing research: Current status and future direction (2012, background): “Additionally, some researchers regard the extrinsic incentives, especially the financial incentives, as an important element in the design of crowd-sourcing contest systems (Kleeman et al. 2008; Mason and Watts 2010).”
Reputation-based incentive protocols in crowdsourcing applications (2011, background): “More concretely, a requester needs to establish sufficient rewards to attract workers’ contributions when workers do not solve tasks solely for altruistic motivations [5][6][10][11].”
Stimulating Online Reviews by Combining Financial Incentives and Social Norms (2017, positive): “…to have sufficient power to detect our hypothesized effects in our two experiments, we conducted an a priori power analysis based on the average effect size obtained in past work most relevant to the current study (Cabral and Li 2015; Goldstein et al. 2008; Mason and Watts 2009; Nolan et al. 2008).”
Income Security in the On-Demand Economy: Findings and Policy Lessons from a Survey of Crowdworkers (2016, background): “The Mason and Watts (2010) findings are, however, disputed in worker forums.”
Digital Footprints: Opportunities and Challenges for Online Social Research (2014, background): “Incentives may also not operate as intended because Turkers appear to anchor on payment levels so that paying more makes them believe they deserve more, producing a greater quantity of work but not at a greater level of quality (Mason & Watts 2009).”
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk (2012, neutral): “’’ Paolacci et al. (2010) also reports an MTurk replication of this experiment.”
Conducting Clinical Research Using Crowdsourced Convenience Samples. (2016, background): “Larger financial incentives generally increase the speed of data collection (Berinsky et al. 2012, Heer & Bostock 2010, Mason & Watts 2010) and increase workers’ willingness to engage in and persevere through difficult tasks (Crump et al. 2013).”
Budget-Optimal Task Allocation for Reliable Crowdsourcing Systems (2011, background): “It has been observed that increasing the cost on crowdsourcing platforms does not directly lead to higher quality of the responses (Mason and Watts 2010).”