Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice

By R. Richardson, J. Schultz, Kate Crawford. 2019

Subfields: AI Ethics, Fairness & Society, Social Sciences & Humanities

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In AI Ethics, Fairness & Society

In Social Sciences & Humanities

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “ntage model performance on certain devices [263]. 72 It has been observed that biases in the data-generating process can also drive unfairness in the resulting models learned from this data (see e.g. [150, 338]). For example, suppose training data is based on user interactions with a product which h”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, background): “US criminal justice system despite significant evidence of shortcomings, such as the linking of criminal datasets to patterns of discriminatory policing (Angwin et al., 2016; Lum and Isaac, 2016; Richardson et al., 2019).”
Towards a critical race methodology in algorithmic fairness (2019, background): “edanalysisofthesystem’soutputs.Forexample, Rashida Richardson and colleagues recently published an extensive survey of the "dirty data" used to develop predictive policing systems in the US [94]. The report reveals that in nine out of the thirteen jurisdictions reviewed, predictive policin”
Power to the People? Opportunities and Challenges for Participatory AI (2022, background): “In parallel to the expansion of data enrichment and moderation, problematic applications of machine learning tools in high stakes domains such as criminal justice [10, 57, 73], healthcare [7, 9], and hiring [2, 71] have prompted both researchers, civil society, and regulators to increasingly urge gr”
The ethics of algorithms: key problems and solutions (2020, unknown): “…data used to train algorithms are seldom obtained “accord-ing to any specific experimental design” (Olhede and Wolfe 2018, 3) and are used even though they may be inaccurate, skewed, or systemically biased, offering a poor representation of a population under study (Richardson et al. 2019).”
The Fallacy of AI Functionality (2022, background): “The problems with crime data are well-documented—whether in diﬀerential victim crime reporting rates [10], selection bias based on policing activities [54, 120], dirty data from periods of recorded unlawful policing [158], and more.”
Bias in machine learning - what is it good for? (2020, background): “If such a system would be used to determine the distribution of police presence, a viscous circle may even be created [11,41].”
Accountable Artificial Intelligence: Holding Algorithms to Account (2020, background): “For instance, a recent study in the US context revealed that predictive systems used by police departments in several jurisdictions are likely built on discriminatory and unlawful historical police practices (Richardson, Schultz, and Crawford 2019).”
Conversational AI: Social and Ethical Considerations (2019, background): “Various studies illustrate this, as bias is found in: detecting skin tones in pedestrians [52], predictive policing systems and justice [38], the display of STEM career ads [27], recidivism algorithms [2], politics of search engines [21], medical applications [13], automatic speech recognition…”
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, background): “[104] complicate how dirty data can be influenced by corrupt, biased, or unlawful practices.”
Data in New Delhi's predictive policing system (2020, background): “Richardson et. al have explored the urgent need for accountability mechanisms within predictive policing systems in the United States [22].”
Human-AI Interactions in Public Sector Decision-Making:"Automation Bias"and"Selective Adherence"to Algorithmic Advice (2021, background): “…healthcare, immigration, or education (Busuioc 2021; Calo and Citron 2021; Diakopoulos 2014; Eubanks 2018; Engstrom et al. 2020; O’Neil 2016; Richardson, Schultz, and Crawford 2019; Veale and Brass 2019; Yeung and Lodge 2019), increasingly permeating non-routine and high-stakes aspects of…”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, background): “Several case studies have demonstrated that “dirty data” used in policy-making systems have led to skewed predictions [53].”
Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context (2020, background): “such as fairness, equity, and ethics [55], has been a recurring topic in recent fair-ML research [21, 29, 41, 87, 109].”