Demographics of Mechanical Turk

By Panagiotis G. Ipeirotis. 2010

653 citations recorded by the source corpus.

Subfields: AI Ethics, Fairness & Society, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In AI Ethics, Fairness & Society

In Human-Computer Interaction

Citations in the corpus

CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, background): “Ipeirotis [29] hypothesizes this to be due to the remote nature of the work, which attracts stay-at-home parents and unemployed or underemployed adults, who are more likely to be women.”
AutoMan: a platform for integrating human-based and digital computation (2012, background): “Demographic studies of Mechanical Turk have shown that the majority of workers on Mechanical Turk are located in the United States and in India [17].”
Human Decision Making with Machine Assistance (2019, result): “The skews in education and ethnicity are consistent with the findings from prior studies on crowd-working platform demographics [41, 54, 57].”
An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets (2011, methodology): “Given that crowdsourcing markets such as Mechanical Turk tend to pay very little money and involve relatively low wages (Ipeirotis 2010), external motivations such as increased pay may have less effect than requesters may desire.”
Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction (2018, result): “ividuals (51% hold at least a college degree), and more liberals (57%) than the U.S. population. This educational skew in AMT population is consistent with observations from previous research studies [31, 45, 53]. On the other hand, our SSI respondents were census-representative to Demographic Attri”
Turkers, Scholars, "Arafat" and "Peace": Cultural Communities and Algorithmic Gold Standards (2015, background): “Along the same lines, researchers have looked at the demographics of workers on AMT [22, 29], but have not focused on the effect these demographics have on gold standard datasets.”
Being a turker (2014, background): “Interestingly, in most studies on this topic, money is reported as the primary motivator [2, 11, 13, 20].”
Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics (2018, background): “The fact that workers from lower income countries tend to have higher education levels is consistent with the findings of studies on MTurk (e.g., Ipeirotis, 2010b), which found that Indian workers on MTurk tend to have more education than workers from the United States.”
Conducting Clinical Research Using Crowdsourced Convenience Samples. (2016, background): “Indian workers are highly educated (Ipeirotis 2010, Khanna et al. 2010) and as honest as US workers (Suri et al. 2011), yet data are consistently of lower quality (Kazai et al. 2012, Khanna et al. 2010, Shaw et al. 2011).”
The Future of Work in the ‘Sharing Economy’. Market Efficiency and Equitable Opportunities or Unfair Precarisation? (2016, neutral): “Yet, in the reviewed studies focussing on online labour markets examples of concentration of work assignments (if not full blown ‘superstar’ effects) are found and no case of long tail effect is documented (Horton, 2014; Ipeirotis, 2010a, 2010b; Musthag & Ganesan, 2013).”
Data collection in a flat world: the strengths and weaknesses of mechanical turk samples (2013, background): ““MTurk” uses Internet crowd sourcing to connect potential workers with jobs or tasks, called “HITs.””
How Does Your Password Measure Up? The Effect of Strength Meters on Password Creation (2012, background): “Furthermore, one of the primary motivations for part of the MTurk population is financial compensation [17], which differs from real-world moti-”
Data quality of platforms and panels for online behavioral research (2021, neutral): “…al., 2016), replicability of known effects such as the Asian disease problem (e.g., Chandler et al., 2014; Feitosa et al., 2015; Goodman et al., 2013), representativeness of the sample (e.g., Ipeirotis, 2010), and non-naivety of participants to common research stimuli (Chandler & Paolacci, 2017).”
Analyzing the Amazon Mechanical Turk marketplace (2010, background): “For the first question, about the demographics of the workers, past research [5, 6] indicated that the workers that participate on the marketplace are mainly coming from the United States, with an increasing proportion coming from India.”
Demographics and Dynamics of Mechanical Turk Workers (2018, background): “4 The dominance of US and India among the worker population is well-documented in prior studies [17, 22, 26].”
Exploring User Perceptions of Discrimination in Online Targeted Advertising (2017, background): “1 MTurk has been shown to provide adequate data quality, but also to be younger and more educated than the general population [24, 26].”
A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001--2010 (2013, background): “…and online participant marketplaces (such as Amazon Mechanical Turk) tend to have limited cross-linguistic value at the moment of writing (e.g., Ipeirotis, 2010 reports that the majority of Amazon Mechanical Turk participants come from two countries, the US and India, primarily because US…”