Predictive Inequity in Object Detection

By Benjamin Wilson, Judy Hoffman, Jamie H. Morgenstern. 2019

244 citations recorded by the source corpus.

In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones. This work is motivated by many recent examples of ML and vision systems displaying higher error rates for certain demographic groups than others. We annotate an existing large scale dataset which contains pedestrians, BDD100K, with Fitzpatrick skin tones in ranges [1-3] or [4-6]. We then provide an in-depth comparative analysis of performance between these two skin tone groupings, finding that neither time of day nor occlusion explain this behavior, suggesting this disparity is not merely the result of pedestrians in the 4-6 range appearing in more difficult scenes for detection. We investigate to what extent time of day, occlusion, and reweighting the supervised loss during training affect this predictive bias.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

In AI Ethics, Fairness & Society

Citations in the corpus

Understanding and Evaluating Racial Biases in Image Captioning (2021, methodology): “, pedestrian detection [73] and facial recognition [16].”
A Step Toward More Inclusive People Annotations for Fairness (2021, background): “While both datasets can be used for evaluating whether a person detector performs consistently across population slices, BDD100k is limited to self-driving applications.”
Large image datasets: A pyrrhic win for computer vision? (2020, background): “…[47]; object detection systems designed to detect pedestrians display higher error rates for recognition of demographic groups with dark skin tones [87]; and gender classiﬁcation systems show disparities in image classiﬁcation accuracy where lighter-skin males are classiﬁed with the highest…”
Measuring Model Biases in the Absence of Ground Truth (2021, background): “Examples of the negative fairness impacts of machine learning models include decreased pedestrian detection accuracy on darker skin tones [29], gender stereotyping in image captioning [6], and perceived racial identities impacting unrelated labels [27].”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, background): “This predictive inequity has been characterized by higher performance for lower Fitzpatrick skin tones [58].”
Critical Race Theory for HCI (2020, background): “Autonomous cars have been found to mis-identify darker-skinned pedestrians 10 percent more often than lighter-skinned subjects [67, 134].”
Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias (2019, background): “An analysis of state-of-the-art object detection systems found decreased pedestrian detection accuracy on darker skin tones [44].”
Conversational AI: Social and Ethical Considerations (2019, background): “Various studies illustrate this, as bias is found in: detecting skin tones in pedestrians [52], predictive policing systems and justice [38], the display of STEM career ads [27], recidivism algorithms [2], politics of search engines [21], medical applications [13], automatic speech recognition…”
Algorithmic injustice: a relational ethics approach (2021, background): “Some identities (and not others) are recognized as a pedestrian, 44 or ﬁt for an STEM career, 69 or in need of medical care.”
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations (2018, background): “, recognition rates of pedestrians in autonomous vehicles are not equally accurate for all groups of people [32]).”
Quantifying Societal Bias Amplification in Image Captioning (2022, background): “societal bias is a transversal problem that affects a variety of tasks within computer vision, such as facial recognition, with black women having higher error rates than white men [7]; object classification, with kitchen objects being associated with women with higher probabilities than with men [4”
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging (2023, background): “…performance disparities between genders (Bhargava and Forsyth 2019; Wang et al. 2020), ages (Wang et al. 2020), races (Buolamwini and Gebru 2018; Wilson, Hoffman, and Morgenstern 2019), socioeconomic statuses (DeVries et al. 2019), and geographic locations (Shankar et al. 2017a; DeVries et al.…”
A Brief History of Artificial Intelligence: On the Past, Present, and Future of Artificial Intelligence (2019, unknown): “Research has, for example, shown that the sensors used in self-driving cars are better in detecting lighter skin tones than darker ones 14 (due to the type of pictures used to train such algorithms) or that decision-sup-port systems used by judges may be racially biased 15 (since they are based on…”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, background): “…object sub-categories (Zhu et al., 2014), scenes (Zhou et al., 2017), gender (Kay et al., 2015), gender contexts (Burns et al., 2018; J. Zhao et al., 2017), skin tones (Buolamwini and Gebru, 2018; Wilson et al., 2019), geographic locations (Shankar et al., 2017) and cultures (DeVries et al., 2019).”