Discovering Attribute Shades of Meaning with the Crowd

By Adriana Kovashka, Kristen Grauman. 2015

40 citations recorded by the source corpus.

To learn semantic attributes, existing methods typically train one discriminative model for each word in a vocabulary of nameable properties. However, this “one model per word” assumption is problematic: while a word might have a precise linguistic definition, it need not have a precise visual definition. We propose to discover shades of attribute meaning. Given an attribute name, we use crowdsourced image labels to discover the latent factors underlying how different annotators perceive the named concept. We show that structure in those latent factors helps reveal shades, that is, interpretations for the attribute shared by some group of annotators. Using these shades, we train classifiers to capture the primary (often subtle) variants of the attribute. The resulting models are both semantic and visually precise. By catering to users’ interpretations, they improve attribute prediction accuracy on novel images. Shades also enable more successful attribute-based image search, by providing robust personalized models for retrieving multi-attribute query results. They are widely applicable to tasks that involve describing visual content, such as zero-shot category learning and organization of photo collections.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Information Extraction

Vittorio Ferrari

Citations in the corpus

Extreme Clicking for Efficient Object Annotation (2017, background): “Quality control is a common process when crowd-sourcing image annotations [4, 36, 43, 53, 56, 60, 62, 66, 70].”
Automatic Understanding of Image and Video Advertisements (2017, background): “Work on semantic visual attributes describes images beyond labeling the objects in them, e.g. with adjective-like properties such as “furry”, “smiling”, or “metallic” [39, 15, 56, 38, 68, 35, 36, 17, 77, 2, 27].”
Y Qualification Main task Y Return N N Y User qualified ? N Feedback Pass ? Pass ? Annotator training Annotating images (2017, background): “Quality control is a common process when crowd-sourcing image annotations [4, 36, 43, 53, 56, 60, 62, 66, 70].”