Examining CNN representations with respect to Dataset Bias

By Quanshi Zhang, Wenguan Wang, Song-Chun Zhu. 2017

113 citations recorded by the source corpus.

Given a pre-trained CNN without any testing samples, this paper proposes a simple yet effective method to diagnose feature representations of the CNN. We aim to discover representation flaws caused by potential dataset bias. More specifically, when the CNN is trained to estimate image attributes, we mine latent relationships between representations of different attributes inside the CNN. Then, we compare the mined attribute relationships with ground-truth attribute relationships to discover the CNN's blind spots and failure modes due to dataset bias. In fact, representation flaws caused by dataset bias cannot be examined by conventional evaluation strategies based on testing images, because testing images may also have a similar bias. Experiments have demonstrated the effectiveness of our method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

In Evaluation & Benchmarks

Citations in the corpus

Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “nglement of features. Potential values of the explainer: The high interpretability of a middle conv-layer is of great value when the performer network needs to earn trust from people. As discussed in [43], due to potential dataset bias and representation bias, a high accuracy on testing images canno”
Interpreting CNNs via Decision Trees (2018, background): “The study of [39] examined representations of conv-layers and automatically discover potential, biased representations of a CNN due to the dataset bias.”
Interpretable Convolutional Neural Networks (2017, background): “As discussed in [38], considering dataset bias and representation bias, a high accuracy on testing images still cannot ensure a CNN to encode correct representations (e.”
Visual interpretability for deep learning: a survey (2018, methodology): “a CNN that was trained to estimate face attributes), [Zhang et al., 2018b] required people to annotate some ground-truth relationships between attributes, e.”
Counterfactuals uncover the modular structure of deep generative models (2018, background): “…activation patterns for ﬁlters [Zeiler and Fergus, 2014, Dosovitskiy and Brox, 2016], correlation between intermediate feature space and data features [Fong and Vedaldi, 2017, Zhang et al., 2017b] or disentangled patterns detected by ﬁlters to compute an explanatory graph [Zhang et al., 2017a].”
Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images (2020, methodology): “The interpretability of model has recently been significantly advanced by several studies through visualizing the trained CNN models [25], [26].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “Other than main XAI goals for AI experts, machine learning explanations are used for other purposes including detecting dataset bias [220], adversarial attack detection [61], and model failure prediction [146].”
Intrinsic disentanglement: an invariance view for deep generative models (2018, background): “…filters ((Zeiler and Fergus, 2014),(Dosovitskiy and Brox, 2016)), to find correlation between intermediate feature space and data features ((Fong and Vedaldi, 2017),(Zhang et al., 2017b)) or to disentangle patterns detected by various filters to compute an explanatory graph (Zhang et al., 2017a).”