Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons

By James Y. Zou, Kamalika Chaudhuri, Adam Tauman Kalai. 2015

33 citations recorded by the source corpus.

We introduce an unsupervised approach to efficiently discover the underlying features in a data set via crowdsourcing. Our queries ask crowd members to articulate a feature common to two out of three displayed examples. In addition, we ask the crowd to provide binary labels for these discovered features on the remaining examples. The triples are chosen adaptively based on the labels of the previously discovered features on the data set. This approach is motivated by a formal framework of feature elicitation that we introduce and analyze in this paper. In two natural models of features, hierarchical and independent, we show that a simple adaptive algorithm recovers all features with less labor than any nonadaptive algorithm. The savings are as a result of automatically avoiding the elicitation of redundant features or synonyms. Experimental results validate the theoretical findings and the usefulness of this approach.

Subfields: Artificial Intelligence, Computer Science Applications, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

ALICE: Active Learning with Contrastive Natural Language Explanations (2020, background): “Contrastive learning has previously been shown to substantially improve unsupervised learning (Abid et al., 2018), feature learning (Zou et al., 2015), and learning probabilistic models (Zou et al., 2013).”
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems (2016, background): “, [10, 16]), with recent interest in active crowd participation to the development of machine learning algorithms [6, 20, 5].”