Generalized Task Markets for Human and Machine Computation

By Dafna Shahaf, Eric Horvitz. 2010

82 citations recorded by the source corpus.

We discuss challenges and opportunities for developing generalized task markets where human and machine intelligence are enlisted to solve problems, based on a consideration of the competencies, availabilities, and pricing of different problem-solving resources. The approach couples human computation with machine learning and planning, and is aimed at optimizing the flow of subtasks to people and to computational problem solvers. We illustrate key ideas in the context of Lingua Mechanica, a project focused on harnessing human and machine translation skills to perform translation among languages. We present infrastructure and methods for enlisting and guiding human and machine computation for language translation, including details about the hardness of generating plans for assigning tasks to solvers. Finally, we discuss studies performed with machine and human solvers, focusing on components of a Lingua Mechanica prototype.

Subfields: Artificial Intelligence, Computer Science Applications, Computer Vision and Pattern Recognition, Economics & Operations Research, Social Sciences & Humanities

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Computer Vision and Pattern Recognition

In Economics & Operations Research

In Social Sciences & Humanities

Citations in the corpus

GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (2023, background): “There is potential for such models to contribute to economic decision-making at the task level, for instance, by refining methods for task and sub-task allocation between humans and machines (Singla et al., 2015; Shahaf and Horvitz, 2010).”
Combining human and machine intelligence in large-scale crowdsourcing (2012, background): “Previous work on decision-theoretic reasoning for crowdsourcing tasks focused on tasks that can be decomposed into smaller tasks [10], and on workflows that are composed of an improve and verify step, which can be solved via methods that perform short lookaheads [3].”