Faceless Person Recognition: Privacy Implications in Social Media

By Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele. 2016

172 citations recorded by the source corpus.

As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy. This works contributes to the understanding of privacy implications of such data sharing by analysing how well people are recognisable in social media data. To facilitate a systematic study we define a number of scenarios considering factors such as how many heads of a person are tagged and if those heads are obfuscated or not. We propose a robust person recognition system that can handle large variations in pose and clothing, and can be trained with few training samples. Our results indicate that a handful of images is enough to threaten users’ privacy, even in the presence of obfuscation. We show detailed experimental results, and discuss their implications.

Subfields: Computer Vision and Pattern Recognition, Privacy & Security, AI Ethics, Fairness & Society, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Privacy & Security

In AI Ethics, Fairness & Society

In Signal Processing

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “For instance, a few images are enough to threaten users’ privacy even in the presence of image obfuscation [420] , and the model parameters of a DNN can be exposed by simply performing input queries on the model [356,357] .”
Defeating Image Obfuscation with Deep Learning (2016, background): “Concurrently with our work, Oh et al. released a preprint [33] where they use a neural network to recognize untagged individuals in social-media images.”
A Hybrid Model for Identity Obfuscation by Face Replacement (2018, methodology): “ive subset, as shown in Table 1. The complete results can be found in the supplementary material. In order to blur the face and hair regions in the input images, we use the same Gaussian kernel as in [1,7]. Note that in contrast to those methods, our reconstructed face model provides the segmentatio”
Examining the Impact of Blur on Recognition by Convolutional Networks (2016, background): “trained for face recognition [10, 17, 4], while Ullman et al.”
A Study of Face Obfuscation in ImageNet (2021, background): “Their effectiveness for privacy protection can be ascertained only empirically, which has been the focus of prior work (Oh et al., 2016; Li et al., 2017; Hasan et al., 2018) but is beyond the scope of this paper.”
I Know That Person: Generative Full Body and Face De-identification of People in Images (2017, background): “[23] propose a system for “faceless” person recognition and show that individuals in a social media setting can be identified even if their faces are blacked out and their clothing varies across images.”
Natural and Effective Obfuscation by Head Inpainting (2017, methodology): “It has thus proved to be relatively immune to common obfuscation techniques like blacking or blurring head regions [17].”
Adversarially Learned Representations for Information Obfuscation and Inference (2019, background): “There is significant prior work in related topics such as visual privacy through image obfuscation (McPherson et al., 2016; Oh et al., 2016; 2017; Brkic et al., 2017; Raval et al., 2017; Wu et al., 2018) and inpanting (Sun et al.”
Designing a Moral Compass for the Future of Computer Vision Using Speculative Analysis (2017, unknown): “The potential for privacy issues appears pervasive given work currently emerging in CV research such as inferring health metrics from social media images [35] or de-anonymizing blurred images [45].”
Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images (2017, unknown): “Apart from detecting attributes, some works introduce new privacy challenges in vision such as adversarial perturbations [31, 35], privacy-preserving video capture [1, 36, 33, 37], person re-identification [2, 30], avoiding face detection [46, 16], full body re-identification [34] and privacy-sensit”
Understanding and Controlling User Linkability in Decentralized Learning (2018, methodology): “All images in both datasets are publicly available on Flickr and have been used in various prior works [44, 51, 63, 72, 83].”
Person Recognition in Personal Photo Collections (2017, unknown): “We discuss and compare against these works in greater detail in Section 5 Some works have exploited the photoalbum metadata, allowing the model to reason over different photos [65], [66].”
Olympus: Sensor Privacy through Utility Aware Obfuscation (2018, unknown): “Rather than considering all powerful attackers, we limit the ML attacker to be one from a known hypothesis class (e.g. a DNN with a known structure).”
When Machine Learning Meets Privacy (2020, background): “Second, traditional protection schemes such as obfuscation no longer work effectively [96, 109].”
Adversarial Image Perturbation for Privacy Protection A Game Theory Perspective (2017, background): “Classical obfuscation techniques, such as face blurring and pixellisation, is not only unpleasant but also ineffective against convnet-based recognisers [28, 37, 18].”
Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study (2018, background): “However, extensive research showed that such an empirical means can be easily hacked too [37,32].”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, methodology): “his makes them ideal to train a classifier on examples resembling usergenerated data on clients. All images in both datasets are publicly available on Flickr and have been used in various prior works [44, 51, 63, 72, 83]. 7.1 PIPA PIPA [82] is a large-scale dataset of ∼37k personal photos uploaded b”
Knockoff Nets: Stealing Functionality of Black-Box Models (2018, background): “ries? 4.What makes for a good knockoff architecture? 2. Related Work Privacy, Security and Computer Vision. Privacy has been largely addressed within the computer vision community by proposing models [28,30,31,42,49,50] which recognize and control privacy-sensitive information in visual content. The”