A Study of Face Obfuscation in ImageNet

By Kaiyu Yang, Jacqueline Yau, Li Fei-Fei, Jia Deng, Olga Russakovsky. 2021

167 citations recorded by the source corpus.

Face obfuscation (blurring, mosaicing, etc.) has been shown to be effective for privacy protection; nevertheless, object recognition research typically assumes access to complete, unobfuscated images. In this paper, we explore the effects of face obfuscation on the popular ImageNet challenge visual recognition benchmark. Most categories in the ImageNet challenge are not people categories; however, many incidental people appear in the images, and their privacy is a concern. We ﬁrst annotate faces in the dataset. Then we demonstrate that face obfuscation has minimal impact on the accuracy of recognition models. Concretely, we benchmark multiple deep neural networks on obfuscated images and observe that the overall recognition accuracy drops only slightly ( ≤ 1 . 0% ). Further, we experiment with transfer learning to 4 downstream tasks (object recognition, scene recognition, face attribute classiﬁcation, and object detection) and show that features learned on obfuscated images are equally transferable. Our work demonstrates the feasibil-ity of privacy-aware visual recognition, improves the highly-used ImageNet challenge benchmark, and suggests an important path for future visual datasets. Data and code are available at https: //github.com/princetonvisualai/imagenet-face-obfuscation .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Privacy & Security

Citations in the corpus

Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Proposed solutions to dataset bias include new data collection approaches [38, 35], manual data cleanup [74, 75], synthetic data generation [58, 20, 60] – or, in extreme cases, even withdrawing the dataset after insurmountable biases have been identifed [13].”
Multimodal datasets: misogyny, pornography, and malignant stereotypes (2021, methodology): “These critiques have resulted in some corrective measures including the retraction of the MS Celeb and TinyImages 3 datasets, blurring of the images of people [14] and ﬁltering out of constituent images to create a sanitized version of the original dataset.”
Maximum Likelihood Training of Score-Based Diffusion Models (2021, background): “It is well-known that ImageNet contains some personal sensitive information and may cause privacy concern [57].”
Generalized Category Discovery (2022, background): “Some of these datasets contain Internet images that feature, often incidentally, people — see [2, 32, 44] for an in-depth discussion of the privacy implications.”
Concept Generalization in Visual Representation Learning (2020, background): “68% drop observed on popular models [71].”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, background): “[103], people’s faces are blurred when they appear “incidentally.”
Generative artificial intelligence: a systematic review and applications (2024, unknown): “Among these, two notable datasets stand out: ImageNet [147], ClebA [82], and in the field of medical science: MIMIC [54], BRATS [86], FastMRI [129] and ChestX-ray [139] Each of these datasets is uniquely designed to challenge and evaluate the models’ abilities to accurately and effectively…”
Towards Robust and Reproducible Active Learning using Neural Networks (2020, background): “Further, ImageNet has privacy concerns due to unblurred faces [31].”
DataComp: In search of the next generation of multimodal datasets (2023, methodology): “[143], obfuscating faces has little impact on model performance, as we also observe in our experiments (Appendix G).”
Intriguing Properties of Vision Transformers (2021, methodology): “In future, we will use a recent ImageNet version which addresses the above issues [48].”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, background): “For example, for an object recognition benchmark, one may lean toward removing or obfuscating people that occur in images since the occurrence of people is largely incidental to the scientiﬁc goals of the dataset (Prabhu and Birhane, 2020; K. Yang et al., 2021).”
RedCaps: web-curated image-text data created by the people, for the people (2021, background): “Privacy: The individual who posts a given photo on Reddit may not be the person appearing in said photo; this can pose privacy risks for people who did not expect to appear in images online [49, 50].”