CrowdForge: crowdsourcing complex work

By Aniket Kittur, Boris Smus, Robert E. Kraut. 2011

623 citations recorded by the source corpus.

Subfields: Computer Science Applications, Management Science and Operations Research, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Management Science and Operations Research

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

PromptChainer: Chaining Large Language Model Prompts through Visual Programming (2022, background): “These chains usually take a divide-and-conquer strategy [9].”
Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision (2016, methodology): “An early version of Mechanical Novel allowed crowd workers to set high-level goals by having them brainstorm and vote on an outline, much like Crowdforge [19].”
Cascade: crowdsourcing taxonomy creation (2013, methodology): “We also plan to apply Cascade to manage volunteers in community-sourcing [6, 10].”
A Survey of Crowdsourcing Systems (2011, background): “Some works proposed an idea of coordination among many individuals to complete more complex human computation tasks [52], [44].”
Crowd-Based Personalized Natural Language Explanations for Recommendations (2016, methodology): “In addition to algorithm support, we use a two stage MapReduce [13] work (cid:13)ow to ensure the quality of crowd synthesized explanations.”
Human-powered Sorts and Joins (2011, methodology): “Kittur et al.e [8] described CrowdForge, a MapReduce-style model for large task decomposition and veriﬁcation.”
A survey of the use of crowdsourcing in software engineering (2017, methodology): “…6http://en.wikipedia.org/wiki/crowdsourcing 7http://www.merriam-webster.com/info/newwords11.htm 8http://www.merriam-webster.com/dictionary/crowdsourcing tasks can be as finely grained as photo tagging or as coarsely grained as software development (Kittur et al., 2011; Xiao and 210 Paik, 2014).”
Context Trees: Crowdsourcing Global Understanding from Local Views (2014, background): “CrowdForge shards the problem into parallelizable tasks and then aggregates their output into a meaningful single outcome, e.g., an article (Kittur et al. 2011).”
Framing Creative Work: Helping Novices Frame Better Problems through Interactive Scaffolding (2021, background): “For crowds, tasks can be decomposed into sub-tasks [34].”
Crowdsourced Data Management: Industry and Academic Perspectives (2015, methodology): “At a higher level of abstraction, toolkits such as CrowdForge [116] and Jabberwocky [22], and Automan [32] help specify crowdsourced workflows.”
The jabberwocky programming environment for structured social computing (2011, methodology): “On top of Dormouse, we built ManReduce, a programming framework inspired by MapReduce [4] (and related to Crowd-Forge [13]) to facilitate complex data processing tasks.”
Being a turker (2014, background): “[14, 15, 17, 18], and this has in turn been highlighted [6, 16].”
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality (2016, background): “Our analysis solely focuses on data labeling microtasks, and we have not yet studied whether our findings translate over to more complex tasks, such as designing an advertisement or editing an essay [3, 25].”
Challenges in Data Crowdsourcing (2016, background): “This human-human region in the space, is still active in modern days: systems like Turkomatic [29], Crowd- Forge [28], and Flash Teams [42], involve humans in the design of the tasks to be solved by human workers.”
Algorithms at Work: The New Contested Terrain of Control (2020, background): “…be replaced, algorithmic replacement can be more far-reaching, especially on on-demand platforms, which allow for the recruiting of workers globally as well as up and down the occupational hierarchy (Aneesh, 2009; Kittur, Smus, Khamkar, & Kraut, 2011; Retelny et al., 2014; Valentine et al., 2017).”
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Accuracy (2016, background): “Our analysis solely focuses on data labeling microtasks, and we have not yet studied whether our findings translate over to more complex tasks, such as designing an advertisement or editing an essay [3, 24].”