A Deeper Look at Dataset Bias

By Tatiana Tommasi, Novi Patricia, Barbara Caputo, Tinne Tuytelaars. 2015

355 citations recorded by the source corpus.

The presence of a bias in each image data collection has recently attracted a lot of attention in the computer vision community showing the limits in generalization of any learning method trained on a specific dataset. At the same time, with the rapid development of deep learning architectures, the activation values of Convolutional Neural Networks (CNN) are emerging as reliable and robust image descriptors. In this paper we propose to verify the potential of the DeCAF features when facing the dataset bias problem. We conduct a series of analyses looking at how existing datasets differ among each other and verifying the performance of existing debiasing methods under different representations. We learn important lessons on which part of the dataset bias problem can be considered solved and which open questions still need to be tackled.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Evaluation & Benchmarks, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Evaluation & Benchmarks

In AI Ethics, Fairness & Society

Citations in the corpus

Does Object Recognition Work for Everyone? (2019, background): “Other studies have revealed biases in computer-vision datasets that allow models to recognize from which dataset an image originated [43, 44].”
Diversity in Faces (2019, background): “Tommasi et al. [47] and Hoﬀman et al. [48] conducted a similar evaluation with deep features, showing that data set bias can be reduced but not eliminated.”
Few-Shot Adversarial Domain Adaptation (2017, background): “Naively training a classifier on one dataset for testing on another is known to produce sub-optimal results, because an effect known as dataset bias [42, 57, 56] or covariate shift [51] occurs due to a difference in the distributions of the images between the datasets.”
Unified Deep Supervised Domain Adaptation and Generalization (2017, neutral): “Visual recognition algorithms are trained with data from a source domain, and when they are tested on a target domain with marginal distribution that differs from the one of the sources, we experience the visual domain adaptation (DA) problem (also known as dataset bias [48, 59, 58], or covariate sh”
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings (2018, background): “Image datasets are known to contain biases that cause models to generalize poorly to new, unseen data [23, 24].”
Revisiting Batch Normalization For Practical Domain Adaptation (2016, background): “In the field of deep learning, feature transferability across different domains is a tantalizing yet generally unsolved topic (Yosinski et al. 2014; Tommasi et al. 2015).”
Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes (2019, background): “from the same underlying distribution as usually assumed [41, 33, 34, 10, 17].”
Face Recognition: Too Bias, or Not Too Bias? (2020, background): “In essence, the source varies, whether from humans [40], data or label types [35], ML models [3, 19], or evaluation protocols [34].”
Synthetic Data for Deep Learning (2019, background): “• finally, even in the presence of a large dataset it often contains biases in its composition of genders, races, or other parameters, sometimes famously so [327,591].”
Challenges and Prospects in Vision and Language Research (2019, background): “Dataset Bias Dataset bias is a serious challenge faced by both computer vision (Torralba and Efros, 2011; Tommasi et al., 2017) and NLP (Bolukbasi et al.”
Learning and the Unknown: Surveying Steps toward Open World Recognition (2019, background): “However, the non-OSR property is a likely source of “negative” dataset bias (Tommasi et al. 2017) and limits application in the real world where the negative space has near inﬁnite variety of inputs that should be rejected.”
Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning (2019, background): “…such as audio collection, content moderation, sentiment analysis, among other tasks where subjective judgments from humans are needed [3, 6, 8, 26, 34], and may be perpetuated via transfer learning [19, 27, 39], a popular practice in deep learning where a model can be repurposed and reused [27].”
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations (2018, background): “More recently, Tommasi et al [29] provided a fresher perspective on this issue using deep learning models.”
REPAIR: Removing Representation Bias by Dataset Resampling (2019, background): “This is referred to as dataset bias [32, 30].”
Reducing Network Agnostophobia (2018, background): “[22], where algorithms are often evaluated, it is a likely source of ”negative” dataset bias [37] and does not necessarily hold true in the real world where the negative space has near infinite variety of inputs that need to be rejected.”