Delayed Impact of Fair Machine Learning

By Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt. 2018

534 citations recorded by the source corpus.

Static classification has been the predominant focus of the study of fairness in machine learning. While most models do not consider how decisions change populations over time, it is conventional wisdom that fairness criteria promote the long-term well-being of groups they aim to protect. This work studies the interaction of static fairness criteria with temporal indicators of well-being. We show a simple one-step feedback model in which common criteria do not generally promote improvement over time, and may in fact cause harm. Our results highlight the importance of temporal modeling in the evaluation of fairness criteria, suggesting a range of new challenges and trade-offs.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “, 2019; Bender, 2019), accounting for long-term impacts (Liu et al., 2019; D’Amour et al., 2020) and feedback loops (Ensign et al.”
Beyond Individual and Group Fairness (2020, background): “Recent works have also studied the long-term impact of optimizing fairness criteria in settings with feedback mechanisms [Liu et al., 2018, Hashimoto et al., 2018, Mouzannar et al., 2019, Kannan et al., 2019].”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “2In contrast, lending to someone unable to repay could hurt their credit score [84].”
Fairness Without Demographics in Repeated Loss Minimization (2018, background): “Simultaneous with our work, Liu et al. (2018) analyzed fairness over time in the context of constraint based fairness criteria, and show that enforcing static fairness constraints do not ensure fairness over time.”
The Frontiers of Fairness in Machine Learning (2018, background): “…the equilibrium eﬀects of imposing statistical deﬁnitions of fairness in a model of a labor market [HC18], specifying the functional relationship between classiﬁcation outcomes and quality [LDR + 18], or by considering the eﬀect of a classiﬁer on a downstream Bayesian decision maker [KRZ18].”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “approaches to auditing and prototyping ML systems (cf. [30]), perhaps aided by simulation tools that can help developers anticipate sensitive contexts or real-world impacts of using an ML system (cf. [44, 75]). Addressing Detected Issues Intervieweesrevealedarangeofchallengesandneedsaround debugging”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…Hashimoto et al., 2018); dialogue agents (Li et al., 2017); molecular compound optimization (Cuccarese et al., 2020; Reker, 2020); decision systems (Liu et al., 2018; D’Amour et al., 2020); and adversarial settings like fraud or 12. https://www.ml4ed.org/ malware detection (Rigaki and Garcia,…”
Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics (2020, background): “…on (i.e., classiﬁes or regresses over) data that are static snap-shots of the problem domain, and consequently often ignores the non-linear and dynamically complex nature of society that involves feedback loops and time-delays between actions and impacts (Ensign et al., 2017; Liu et al., 2018).”
The Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning (2018, background): “Recent work has considered the feedback effects that may follow machine learning predictions, paying special attention to how even algorithms that satisfy standard fairness constraints can compound inequality (Hu & Chen, 2018; Liu et al., 2018).”
Fairness with Dynamics (2019, background): “Prior work has studied one-step feedback effects (Liu et al., 2018), but do not propose fair learning algorithms.”
Optimal Decision Making Under Strategic Behavior (2019, result): “mentally different and their technical contributions are orthogonal to ours. More broadly, our work also relates to several recent studies on the long-term consequences of machine learning algorithms [10, 15, 17, 21] and recommender systems [19, 20]. Once we focus on the utility of a decision policy”
Fairness-Aware Programming (2019, background): “For instance, the decisions made may affect the population (e.g., in a setting like giving loans), a problem that has been recently explored by Liu et al. [20].”
Why Machine Learning May Lead to Unfairness: Evidence from Risk Assessment for Juvenile Justice in Catalonia (2019, methodology): “As a future task, we plan to address mitigation methods that consider particular issues of this dataset and the respective domain.”
Robust Optimization for Fairness with Noisy Protected Groups (2020, background): “[45] show that the long-term impacts may not always be desirable, and the choice of criteria should be heavily influenced by domain experts, along with awareness of tradeoffs.”
Fair Generative Modeling via Weak Supervision (2019, background): “, 2019; Jo & Gebru, 2020) or algorithmic decisions such as the loss function or evaluation metric (Hardt et al., 2016; Buolamwini & Gebru, 2018; Kim et al., 2018; Liu et al., 2018; Hashimoto et al., 2018), among others, may have undesirable consequences.”
Implicit Diversity in Image Summarization (2019, background): “Uninformed application of any technology that aims to ensure fairness can inadvertently cause more harm than good [9, 54, 89].”
The Social Cost of Strategic Classification (2018, background): “a loan) to a negative individual (someone who will default) can result in a long-term negative impact to that individual [21].”