The Social Cost of Strategic Classification

By Smitha Milli, John Miller, Anca Dragan, Moritz Hardt. 2018

212 citations recorded by the source corpus.

Consequential decision-making typically incentivizes individuals to behave strategically, tailoring their behavior to the specifics of the decision rule. A long line of work has therefore sought to counteract strategic behavior by designing more conservative decision boundaries in an effort to increase robustness to the effects of strategic covariate shift. We show that these efforts benefit the institutional decision maker at the expense of the individuals being classified. Introducing a notion of social burden, we prove that any increase in institutional utility necessarily leads to a corresponding increase in social burden. Moreover, we show that the negative externalities of strategic classification can disproportionately harm disadvantaged groups in the population. Our results highlight that strategy-robustness must be weighed against considerations of social welfare and fairness.

Subfields: General Decision Sciences, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Decision Sciences

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “, 2020) and feedback loops (Ensign et al., 2018; Milli et al., 2019; Martin Jr. et al., 2020).”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “While it may be possible to make predictions robust to strategic behavior, this may also have social costs, especially for disadvantaged groups [59, 90].”
Actionable Recourse in Linear Classification (2018, background): “These include: inverse classification, which aims to determine how the inputs to a model can be manipulated to obtain a desired outcome [1, 10]; strategic classification, which considers how to build classifiers that are robust to malicious manipulation [13, 16, 21, 22, 30]; adversarial perturbation”
Fairness in Machine Learning: A Survey (2020, background): “Subsequent publications expanded this view of recommender fairness by proposing new metrics and algorithms [12, 29, 59, 272, 303, 317].”
Fairness is not static: deeper understanding of long term fairness via simulation studies (2020, background): “[22] point out that deploying a robust decision rule in this setting has important fairness implications.”
The Disparate Effects of Strategic Manipulation (2018, background): “[21] also consider the social impacts of strategic classification.”
Optimal Decision Making Under Strategic Behavior (2019, background): “With this motivation, there has been a recent ﬂurry of work on strategic classiﬁcation [1, 2, 5, 6, 8, 11, 16].”
Performative Prediction (2020, background): “Recent work in this area considers issues of incentive design (Kleinberg & Raghavan, 2019; Miller et al., 2020), control over an algorithm (Burrell et al., 2019), and fairness (Hu et al., 2019; Milli et al., 2019).”
Explainable AI: A Review of Machine Learning Interpretability Methods (2020, background): “[83] also studied how individuals adjust their behaviour strategically to manipulate decision rules in order to gain favourable treatment from decision-making models.”
How Do Classifiers Induce Agents to Invest Effort Strategically? (2018, background): “in the case of spam detection, and more recently in terms of Stackelberg competitions, in which the evaluator publishes a rule and the agent may respond by manipulating their features strategically [5, 13, 18, 23, 29].”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Milli et al. (2018a) explore the tradeoffs between robustness to manipulation and fairness and other social welfare considerations.”