Context Trees: Crowdsourcing Global Understanding from Local Views

By Vasilis Verroios, Michael S. Bernstein. 2014

30 citations recorded by the source corpus.

Crowdsourcing struggles when workers must see all of the pieces of input to make an accurate judgment. For example, to find the most important scenes in a novel or movie, each worker must spend hours consuming the entire plot to acquire a global understanding and then apply that understanding to each local scene. To enable the crowdsourcing of large-scale goals with only local views, we introduce context trees, a crowdsourcing workflow for creating global summaries of a large input. Context trees recursively combine elements through written summaries to form a tree. Workers can then ground their local decisions by applying those summaries back down to the leaf nodes. In the case of scale ratings such as scene importance, we introduce a weighting process that percolates ratings downwards through the tree so that important nodes in unimportant branches are not overweighted. When using context trees to rate the importance of scenes in a 4000-word story and a 100-minute movie, workers’ ratings are nearly as accurate as those who saw the entire input, and much improved over the traditional approach of splitting the input into independent segments. To explore whether context trees enable crowdsourcing to undertake new classes of goals, we also crowdsource the solution to a large hierarchical puzzle of 462,000 interlocking pieces.

Subfields: Artificial Intelligence, Computer Science Applications, Human-Computer Interaction, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Human-Computer Interaction

In Signal Processing

Citations in the corpus

AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (2021, background): “Meanwhile, Context Trees [65] hierarchically summarize and trim the otherwise overwhelming global contexts, making them compact enough for a single worker to digest.”
Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision (2016, background): “Context trees [38] recursively merge subparts of a long story to gather an emergent understanding of the larger plot; this strategy explicitly shifts from looking at low-level input to the larger story structure and vice-versa, but does not allow workers to modify the story or summary.”
Examining Crowd Work and Gig Work Through the Historical Lens of Piecework (2017, background): “Research has sought to ameliorate this issue by designing workﬂows to help workers “act with global understanding when each contributor only has access to local views” [151], typically by automatically or manually generating higher–level representations for the workers to reﬂect on [34, 151, 80].”