ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism

By Pierre Stock, Moustapha Cissé. 2017

48 citations recorded by the source corpus.

ConvNets and Imagenet have driven the recent success of deep learning for image classification. However, the marked slowdown in performance improvement, the recent studies on the lack of robustness of neural networks to adversarial examples and their tendency to exhibit undesirable biases (e.g racial biases) questioned the reliability and the sustained development of these methods. This work investigates these questions from the perspective of the end-user by using human subject studies and explanations. We experimentally demonstrate that the accuracy and robustness of ConvNets measured on Imagenet are underestimated. We show that explanations can mitigate the impact of misclassified adversarial examples from the perspective of the end-user and we introduce a novel tool for uncovering the undesirable biases learned by a model. These contributions also show that explanations are a promising tool for improving our understanding of ConvNets' predictions and for designing more reliable models

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Exploring the Limits of Weakly Supervised Pretraining (2018, methodology): “Furthermore, following [32], we measure the rectified classification accuracy of this model on val-IN-1k.”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “has found that model predictions directly reflect the biases found in training datasets, such as image classifiers learning to associate ethnicity with specific activities (Stock and Cisse, 2017).”
Does Object Recognition Work for Everyone? (2019, neutral): “ of Google Image Search revealed exaggeration of stereotypes and systematic underrepresentation of women in search results [26], and a study of ImageNet revealed correlations between classes and race [40]. Other studies have revealed biases in computer-vision datasets that allow models to recognize ”
Women also Snowboard: Overcoming Bias in Captioning Models (2018, background): “One common theme is the notion of bias amplification, in which bias is not only learned, but amplified [7,6,4].”
Deep Clustering for Unsupervised Learning of Visual Features (2018, background): “However, Stock and Cisse [7] have recently presented empirical evidence that the performance of state-of-the-art classiﬁers on ImageNet is largely underestimated, and little error is left unresolved.”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, methodology): “On the left, the test image that is processed by the DNN is the same than the one used by Stock and Cissé [105].”
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2017, background): “For instance, TCAV provides quantitative con-ﬁrmations to the qualitative ﬁndings from (Stock & Cisse, 2017) that found ping-pong balls are highly correlated with a particular race.”
Causal Generative Neural Networks (2017, background): “However, they often mistake correlation for causation [Stock and Cisse, 2017], which can have catastrophic consequences for agents that plan and decide from observation.”
Troubling Trends in Machine Learning Scholarship (2018, background): “misclassifying “Asians dressed in red” as ping-pong balls [73].”
SoK: Security and Privacy in Machine Learning (2018, methodology): “For instance, connections between fairness and security have recently been drawn; techniques like adversarial example algorithms were applied to estimate how representative of a class a particular input is, which led to the identification of racial biases in popular image datasets [109].”
Algorithmic Fairness (2020, background): “Second, the labels of the data usually depend on annotators and are not always accurate [129, 134].”
Gradient Matching for Domain Generalization (2021, unknown): “…and Lopez-Paz, 2020; Ganin et al., 2016), a key aspect of domain generalization is to learn from features that remain invariant across multiple domains, while ignoring those that are spuriously correlated to label information (as deﬁned in Torralba and Efros (2011); Stock and Cisse (2017)).”
Feature-Wise Bias Amplification (2018, background): “…also potentially concerning as it relates to the fairness of a model’s predictions (Zhao et al., 2017; Burns et al., 2018; Bolukbasi et al., 2016; Stock & Ciss´e, 2017) as models that learn to overpredict negative outcomes for certain groups may exacerbate stereotypes, preju-dices, and…”
Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring (2018, background): “For instance, a fair face recognition system should achieve the same recognition rate irrespective of the gender or ethnicity of the subjects [40, 36].”
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications (2019, methodology): “To compare prototypicality, the work of Stock & Cisse (2017) that inspired our current work used a simple and efficient `∞-based adversarial-example attack based on an iterative gradient descent introduced by Kurakin et al. (2016).”
A Review on Fairness in Machine Learning (2022, background): “For example, Stock and Cisse [193] show that human annotators fail to capture parts of the visual concepts in images, and Van Miltenburg [202] show that the crowdsourced descriptions of the images in the Flickr30K dataset are often based on stereotypes and prejudices of the annotators; another chall”
A Systematic Study of Bias Amplification (2022, background): “Bias amplification is concerning as it can foster the proliferation of undesired stereotypes [12, 41, 46, 47] or lead to unjustifiable differences in model accuracy between subgroups of users [7, 11].”
xGEMs: Generating Examplars to Explain Black-Box Models (2018, methodology): “In terms of generating examples, adversarial criticisms [35] and the class of generative networks like GANs are relevant approaches.”