Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy

By Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, Olga Russakovsky. 2019

357 citations recorded by the source corpus.

Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including offensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the first steps to mitigate them constructively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

Segment Anything (2023, methodology): “We repeat the analysis for perceived age in Table 2 (bottom left), noting that those who are perceived to be younger and older have been shown to be underrepresented in large-scale datasets [110].”
A Step Toward More Inclusive People Annotations for Fairness (2021, background): “With our updated annotations, we provide the attribute labels similar to those annotated at the image level in ImageNet.”
Multimodal datasets: misogyny, pornography, and malignant stereotypes (2021, methodology): “For example, the curators of Imagenet advocated removing 2674 out of 2832 existing synsets in the person subtree of the label space [12].”
SWAD: Domain Generalization by Seeking Flat Minima (2021, background): “Even in a simple image recognition scenario, systems rely on wrong cues for their prediction, e.g. , geographic distribution [3], demographic statistics [4], texture [5], or backgrounds [6].”
Large image datasets: A pyrrhic win for computer vision? (2020, background): “Acknowledging some of the problems, the authors from the ImageNet team did recently attempt to address [91] the stagnant concept vocabulary of WordNet.”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, background): “…proliferation of algorithmic fairness interventions have emerged in recent years that hinge on balancing representation of different demographic groups within training data-sets — the data used for algorithms to “ learn ” associations (Shankar et al., 2017; Merler et al., 2019; Yang et al., 2020).”
A Study of Face Obfuscation in ImageNet (2021, background): “For ImageNet, researchers have examined issues such as geographic diversity, the category vocabulary, and imbalanced representation (Shankar et al., 2017; Stock & Cisse, 2018; Dulhanty & Wong, 2019; Yang et al., 2020).”
Fair Attribute Classification through Latent Space De-biasing (2020, background): “Works such as [9, 43] propose techniques for better sampling data to more accurately represent all people.”
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision (2022, methodology): “Further, studies [136] show that training on ImageNet might lead to potential biases and harms in models, that are then transferred to the downstream tasks that model is applied on.”
Distilling Model Failures as Directions in Latent Space (2022, neutral): “Moreover, identifying meaningful subpopulations within data allows for dataset reﬁnement (such as ﬁltering or relabeling) (Yang et al., 2019; Stock & Cisse, 2018), and training more fair (Kim et al., 2019; Du et al., 2021) or accurate (Jabbour et al., 2020; Srivastava et al., 2020) models.”
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases (2020, background): “[78] show that the ImageNet categories unequally represent of race and gender; for instance, the “groom" category may contain mostly White people.”
Quantifying Societal Bias Amplification in Image Captioning (2022, background): “Detecting and addressing these biases, often known as societal biases, has become an active research direction in our community [1, 11, 21, 30, 32, 37, 44].”
Towards Robust and Reproducible Active Learning using Neural Networks (2020, methodology): “Societal impact and Limitations: For some of our experiments, we use ImageNet data, which lacks gender and ethnic diversity [30].”
Concept Generalization in Visual Representation Learning (2020, neutral): “It was recently shown that a subset of IN-21K categories might exhibit undesirable behavior in downstream computer vision applications [66].”
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? (2022, background): “[50] argued towards fairer datasets and filter parts of ImageNet.”