Fair Work: Crowd Work Minimum Wage with One Line of Code

By Mark E. Whiting, Grant Hugh, Michael S. Bernstein. 2019

94 citations recorded by the source corpus.

Accurate task pricing in microtask marketplaces requires substantial effort via trial and error, contributing to a pattern of worker underpayment. In response, we introduce Fair Work, enabling requesters to automatically pay their workers minimum wage by adding a one-line script tag to their task HTML on Amazon Mechanical Turk. Fair Work automatically surveys workers to find out how long the task takes, then aggregates those self-reports and auto-bonuses workers up to a minimum wage if needed. Evaluations demonstrate that the system estimates payments more accurately than requesters and that worker time surveys are close to behaviorally observed time measurements. With this work, we aim to lower the threshold for pro-social work practices in microtask marketplaces.

Subfields: Computer Science Applications, Management of Technology and Innovation, Sociology and Political Science, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Management of Technology and Innovation

In Sociology and Political Science

In Evaluation & Benchmarks

Citations in the corpus

CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, methodology): “We use the Fair Work tool (Whiting et al., 2019) to ensure a pay rate of at least $15/hour.”
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning (2021, methodology): “Human evaluation tasks: To validate the correctness of our question-answer generation process and determine the percent of questions in of our dataset that include these errors, we run human validation tasks on Amazon Mechanical Turk and pay $15 USD per hour.”
Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options (2020, methodology): “each of our HITs (Whiting et al., 2019).”
On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices (2021, background): “One aspect of this regards to accurate and fair task pricing [79], which previous work has shown to impact the number of contributions produced by workers [47], the quality of their work [18], and being a relevant aspect from an ethical perspective.”
Benchmarking Large Language Models for News Summarization (2023, methodology): “Therefore, we pay our writers $4 for every article they summarize to following the recommended practice (Whiting et al., 2019).”