Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

By Mohsan S. Alvi, Andrew Zisserman, Christoffer Nellåker. 2018

270 citations recorded by the source corpus.

Neural networks achieve the state-of-the-art in image classification tasks. However, they can encode spurious variations or biases that may be present in the training data. For example, training an age predictor on a dataset that is not balanced for gender can lead to gender biased predicitons (e.g. wrongly predicting that males are older if only elderly males are in the training set). We present two distinct contributions: (1) An algorithm that can remove multiple sources of variation from the feature representation of a network. We demonstrate that this algorithm can be used to remove biases from the feature representation, and thereby improve classification accuracies, when training networks on extremely biased datasets. (2) An ancestral origin database of 14,000 images of individuals from East Asia, the Indian subcontinent, sub-Saharan Africa, and Western Europe. We demonstrate on this dataset, for a number of facial attribute classification tasks, that we are able to remove racial biases from the network feature representation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “Studies in algorithmic fairness have focused on either 1) discovering (auditing) existing bias in datasets or systems [50, 7, 30], 2) making a better dataset [39, 1], or 3) designing a better algorithm or model [12, 1, 47, 71, 70].”
A Step Toward More Inclusive People Annotations for Fairness (2021, background): “, Celeb-A [18], FairFace [14], UTKFace [32]) but may also be useful for studying bias [1, 23].”
DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “A popular category of techniques involve adversarial representation learning [34, 29, 35] to mitigate the impact of the bias attribute on the task attribute.”
Unsupervised label noise modeling and loss correction (2019, background): “This overﬁtting also arises in biases that networks encounter during training, e.g., when a dataset contains class imbalances (Alvi et al., 2018).”
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging (2023, background): “This suggests that they are inappropriately using these notions as the basis for tagging behaviors (Alvi, Zisserman, and Nellaker 2018; Roberts 2018; Song and Shmatikov 2020; Wang et al. 2019).”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “Some other related studies in this area work directly over face representations or deep face models by eliminating there undesired or protected information like identity, gender, or facial expressions [179]–[181].”
A Survey of Deep Learning for Scientific Discovery (2020, background): “A line of active research studies how to debias these representations [4, 241].”
FlowSAN: Privacy-Enhancing Semi-Adversarial Networks to Confound Arbitrary Face-Based Gender Classifiers (2019, background): “While GDPR prohibits unsolicited data extraction from European users, the possibility of unlawful data collection still remains and can ultimately lead to negative societal, economic, and political consequences [41]–[43].”
Quantifying Societal Bias Amplification in Image Captioning (2022, background): “Detecting and addressing these biases, often known as societal biases, has become an active research direction in our community [1, 11, 21, 30, 32, 37, 44].”
Towards causal benchmarking of bias in face analysis algorithms (2020, background): “The machine learning community is active in analyzing biases of learning models, and how one may train models where bias is mitigated [3, 14, 18, 31, 33, 41, 46, 51, 68], usually by ensuring that performance is equal across certain subgroups of a dataset.”
Deep Learning for Face Recognition: Pride or Prejudiced? (2019, methodology): “Simultaneously, Alvi et al. [4] proposed a joint learning and unlearning (JLU) technique for eliminating bias from CNN networks.”
Are My Deep Learning Systems Fair? An Empirical Study of Fixed-Seed Training (2021, methodology): “Various debiasing algorithms [3, 47, 59, 98, 99] optimize DL models toward a speciﬁc fairness goal and it is unclear how software implementation aﬀects these debiasing algorithms.”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation (2021, background): “Studies in algorithmic fairness have focused on either 1) discovering (auditing) existing bias in datasets or systems [52, 7, 30, 39, 22], 2) making a better dataset [40, 1], or 3) designing a better algorithm or model [12, 1, 49, 72, 71, 62, 41, 27], typically by learning representations invariant ”
Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network (2018, background): “Although some studies, e.g. unequal-training [9] and suppressing attributes [8, 43, 44, 42], have made effort to mitigate racial and gender bias in several computer vision tasks, this study remains to be vacant in FR.”
Deep Face Recognition: A Survey (2018, methodology): “[284] applied a confusion loss to make a classifier fail to distinguish attributes of examples so that multiple spurious variations are removed from the feature representation.”
Fair Attribute Classification through Latent Space De-biasing (2020, background): “[41] quantitatively compare different techniques for bias mitigation, including weighted training [7, 15], adversarial training with losses inspired by [2, 45], and their proposed domain discriminative and domain independent training.”
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation (2019, methodology): “We experimented with other combinations of layers and losses, including applying the projection method of [51] onto the confusion loss of [1, 46], and achieved similar results.”
Machine learning and deep learning—A review for ecologists (2022, background): “For example, it is possible to algorithmically detect biases (Cirillo et al., 2020 ) and correct the models accordingly (e.g. Alvi et al., 2018 ; Kim et al., 2019 ) by using fairness metrics to guide training so that underrepresented groups are not neglected (e.g. Liu & Vicente, 2021 ).”
Learning Not to Learn: Training Deep Neural Networks With Biased Data (2018, methodology): “[1] proposed joint learning and unlearning method to remove bias from neural network embedding.”