Autotagging Facebook: Social network context improves photo annotation

By Zak Stone, Todd Zickler, Trevor Darrell. 2008

215 citations recorded by the source corpus.

Most personal photos that are shared online are embedded in some form of social network, and these social networks are a potent source of contextual information that can be leveraged for automatic image understanding. In this paper, we investigate the utility of social network context for the task of automatic face recognition in personal photographs. We combine face recognition scores with social context in a conditional random field (CRF) model and apply this model to label faces in photos from the popular online social network Facebook, which is now the top photo-sharing site on the Web with billions of photos in total. We demonstrate that our simple method of enhancing face recognition with social network context substantially increases recognition performance beyond that of a baseline face recognition system.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Faceless Person Recognition: Privacy Implications in Social Media (2016, background): “Various other cues have been explored, for example: attributes classification [40, 41], social context [42, 43], relative camera positions [44], space-time priors [45], and photo-album priors [46].”
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Software systems employing deep neural networks are now applied widely in industry, powering the vision systems in social networks [47] and self-driving cars [5], providing assistance to radiologists [24], underpinning recommendation engines used by online platforms [9, 12], enabling the bestperform”
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale (2015, background): “[26] show that social network’s context improves large scale face recognition.”
Person Recognition in Personal Photo Collections (2015, methodology): “One can also exploit album context, which is a rich source of identity information [16, 34, 35]; however, it was not used in this work for fair comparison.”
What drives successful social networking services? A comparative analysis of user acceptance of Facebook and Twitter (2014, background): “In addition, the IU is the closest factor determining user action.”
Moving beyond untagging: photo privacy in a tagged world (2010, background): “Photo privacy may become even more problematic in the future as researchers are discovering effective automated algorithms to identify people in images and tag them [16].”
Landmark classification in large-scale image collections (2009, background): “For instance, social ties have been found to improve face recognition performance on Facebook [15].”
Boosted multi-task learning for face verification with applications to web image and video search (2009, methodology): “[13] utilized the social network context provided by Facebook to autotag personal photographs on the web.”
Face recognition in unconstrained videos with matched background similarity (2011, background): “To provide researchers with a wider and more arbitrary range of viewing conditions, such as those available on-line [41], the Labeled Faces in the Wild (LFW) [19] and its extension, the Public Figure Face Database (PubFig) [21] sets were assembled.”
Image Labeling on a Network: Using Social-Network Metadata for Image Classification (2012, background): “Friendship information between users was studied for tag recommendation in [20], and in [22] for the case of Facebook.”
Understanding images of groups of people (2009, background): “In [10, 19, 24], co-occurences between individuals in labeled images are considered for reasoning about the identities of groups of people (instead of one person at a time).”
Preserving integrity in online social networks (2020, background): “This body of work traces back to early work since the inception of social networks that considered social context for content classification [36, 75, 87, 111].”