Disparate Impact in Big Data Policing

By Andrew D. Selbst. 2017

184 citations recorded by the source corpus.

Police departments large and small have begun to use data mining techniques to predict the where, when, and who of crime before it occurs. But data mining systems can have a disproportionately adverse impact on vulnerable communities, and predictive policing is no different. Reviewing the technical process of predictive policing, the Article begins by illustrating how use of predictive policing technology will often result in disparate impact on communities of color. After evaluating the possibilities for Fourth Amendment regulation and finding them wanting, the Article turns toward a new regulatory proposal.The Article proposes the use of a rulemaking procedure centered on “discrimination impact assessments.” Predictive policing, like a great deal of data mining solutions, is sold in part as a “neutral” method to counteract unconscious biases. At the moment, however, police departments adopting the technology are not evaluating its potential for a discriminatory impact, which might reproduce or exacerbate the unconscious biases its proponents claim it will cure. Modeled on the environmental impact statements of the National Environmental Policy Act, discrimination impact assessments would require police departments to evaluate the potential discriminatory effects of competing alternative algorithms and to publicly consider mitigation procedures. This regulation balances the need for police expertise in the adoption of new crime control technologies with transparency and public input regarding the potential for harm. Such a public process will also serve to increase trust between police departments and the communities they serve.

Subfields: Economics and Econometrics, Nature and Landscape Conservation, Strategy and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Nature and Landscape Conservation

In Strategy and Management

Citations in the corpus

Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment (2017, background): “For example, Selbst [60] cites media debates on consumer finance [21], employment [70], housing [11], health care [63], and sentencing [67].”
Fairness and Abstraction in Sociotechnical Systems (2019, background): “In fact, a number of recent papers have proposed ideas along these lines [12, 27, 37, 55, 58, 60, 71].”
A Causal Bayesian Networks Viewpoint on Fairness (2018, neutral): “As previous research has shown [29,35,44], modern policing tactics center around targeting a small number of neighborhoods — often disproportionately populated by non-white and low income residents — with recurring patrols and stops.”
FactSheets: Increasing trust in AI services through supplier's declarations of conformity (2019, background): “Selbst [31] has proposed an algorithmic impact statement for AI that follows the form and purpose of EISs.”
The fallacy of inscrutability (2018, background): “Another common approach to dealing with inscrutability is to eschew attempts to understand the system at all and to treat outcomes from computer systems as though they were unforeseeable externalities analogous to pollution [17,18] or other sources of nuisance [19].”
Data in New Delhi's predictive policing system (2020, methodology): “One of the arguments in favor of deploying predictive policing systems is that it leverages technology to free public institutions from human prejudice [24].”
From What to How. An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices (2019, background): “For instance, Selbst, 2017 articulates how the adoption of predictive policing tools is leading to more people of colour being arrested, jailed or physically harmed by police.”
Does Predictive Policing Lead to Biased Arrests? Results From a Randomized Controlled Trial (2018, background): “Selbst [52] also warns that racially biased outcomes may become entrenched in place-based predictive policing given data collected via discriminatory policing practices.”
Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study (2022, background): “For instance, algorithmic bias in AI applications such as facial recognition, predictive policing, and credit score assignment has had negative impacts on communities of color (Buolamwini & Gebru, 2018; Kirkpatrick, 2016; Selbst, 2017; Van Brakel, 2016).”
Fairness in Machine Learning (2020, background): “ProPublica claimed that COMPAS did not satisfy EFPRs and EFNRs, as FPRs = 44 .”
Governing artificial intelligence: ethical, legal and technical opportunities and challenges (2018, unknown): “A growing body of the literature covers questions of AI and ethical frameworks [1,6–10], laws [3,11–14] to govern the impact of AI and robotics [15], technical approaches like algorithmic impact assessments [16–18], and building trustworthiness through system validation [19].”
FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity. (2018, background): “Selbst has proposed an algorithmic impact statement for AI that follows the form and purpose of EISs [31].”
Gender shades : intersectional phenotypic and demographic evaluation of face datasets and gender classifiers (2017, unknown): “For example, following the notice-and-comments process used for environmental impact statements, citizens can weigh in on discriminatory impact assessments (Selbst, 2017).”
Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019, background): “For example, zip code tends to predict race due to a history of segregation [21, 46]; answers to personality tests identify people with disabilities [4, 53]; and keywords can reveal gender on a resume [12].”
From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices (2019, background): “For instance, Selbst (2017) articulates how the adoption of predictive policing tools is leading to more people of colour being arrested, jailed or physically harmed by police Transformative effects Algorithmic activities, like profiling, can lead to challenges for autonomy and informational…”
Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018, background): “Those concerning race have taken centre stage, particularly in the US—ranging from discrimination in recidivism scores used in parole decisions [4, 13] to uneven demographic targeting in systems used for policing on the basis of spatiotemporal crime risk [70].”