Image Labeling on a Network: Using Social-Network Metadata for Image Classification

By Julian McAuley, Jure Leskovec. 2012

193 citations recorded by the source corpus.

Large-scale image retrieval benchmarks invariably consist of images from the Web. Many of these benchmarks are derived from online photo sharing networks, like Flickr, which in addition to hosting images also provide a highly interactive social community. Such communities generate rich metadata that can naturally be harnessed for image classification and retrieval. Here we study four popular benchmark datasets, extending them with social-network metadata, such as the groups to which each image belongs, the comment thread associated with the image, who uploaded it, their location, and their network of friends. Since these types of data are inherently relational, we propose a model that explicitly accounts for the interdependencies between images sharing common properties. We model the task as a binary labeling problem on a network, and use structured learning techniques to learn model parameters. We find that social-network metadata are useful in a variety of classification tasks, in many cases outperforming methods based on image content.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yi Yang
2. Tao Mei
3. Xiaojun Chang
4. Eric P. Xing
5. Zicheng Liao
6. Hexiang Hu
7. Zhiwei Deng
8. Greg Mori

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Yi Yang
3. Xiaogang Wang
4. Dacheng Tao
5. Jitendra Malik
6. Trevor Darrell
7. Ming-Hsuan Yang
8. Bernt Schiele

Citations in the corpus

• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision (2022, methodology): “The Flickr dataset is a graph of images from Flickr (McAuley & Leskovec, 2012; Zeng et al., 2020).”
• Semantic Pooling for Complex Event Analysis in Untrimmed Videos (2016, background): “For instance, [41] proposed a generative model to integrate cues such as scene, object categories and people to segment and recover the event category in a single image, while [42] further exploited user context, location, and user-provided tags and comments on a photo sharing website.”
• Learning from the Past: Meta-Continual Learning with Knowledge Embedding for Jointly Sketch, Cartoon, and Caricature Face Recognition (2020, background): “What’s more, for these images, it seems to be implicitly related [29].”
• Structured Label Inference for Visual Understanding (2018, methodology): “Model mAPL mAPI RL PL RI RI Graphical Model [35] 49.”
• Hierarchical block structures and high-resolution model selection in large networks (2013, background): “[81] Julian McAuley and Jure Leskovec, “Image labeling on a network: Using social-network metadata for image classification,” in Computer Vision – ECCV 2012 , Lecture Notes in Computer Science No.”
• Learning Social Image Embedding with Deep Multimodal Attention Networks (2017, background): “Based on the study of [18] on these collections, we crawl the original images from the Flickr website.”