Equality of Opportunity in Supervised Learning
By Moritz Hardt, Eric Price, Nati Srebro. 2016
5,258 citations recorded by the source corpus.
We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. We enourage readers to consult the more complete manuscript on the arXiv.
Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Yoshua Bengio
- Jianfeng Gao
- Quoc V. Le
- Sergey Levine
- Yejin Choi
- Ruslan Salakhutdinov
In Safety Research
In Safety, Robustness & Interpretability
- Percy Liang
- Yulia Tsvetkov
- Silvia Chiappa
- Dana Pessach
- Erez Shmueli
- Adrian Weller
- Kai-Wei Chang
- Nicholas Carlini
Citations in the corpus
- Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, methodology): “To quantify sentiment bias, we propose the use of individual and group fairness metrics from the fair machine learning literature (Dwork et al., 2012; Jiang et al., 2019; Hardt et al., 2016).”
- Model Cards for Model Reporting (2018, background): “of Opportunity, and equal false negative and false positive rates across groups is equivalent to fulfilling Equality of Odds [26].”
- Societal Biases in Language Generation: Progress and Challenges (2021, background): “Language generation tasks often involve stochastic generation of open-ended and lengthy texts, traits that are not directly compatible with traditional algorithmic bias definitions (e.g., equalized odds, equal opportunity, demographic parity (Dwork et al., 2012; Hardt et al., 2016)).”
- Shortcut learning in deep neural networks (2020, background): “Individual fairness aims at treating similar individuals similarly while group fairness aims at treating subgroups no different than the rest of the population [133, 134].”
- Measurement and Fairness (2019, neutral): “5Error-rate balance is also known as equalized odds [28].”
- The Risk of Racial Bias in Hate Speech Detection (2019, background): “These discrepancies in FPR across groups violate the equality of opportunity criterion, indicating discriminatory impact (Hardt et al., 2016).”
- On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “AO can be empirically interpreted as the average disparity between separately calculated true positive rates (TPR) and false positive rates (FPR) of various groups [26].”
- FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “This will raise ethical concerns about fairness of automated systems, which has emerged as a critical topic of study in the recent machine learning and AI literature [17, 11].”
- Does Object Recognition Work for Everyone? (2019, background): “The equalized odds [18] fairness principle (also referred to as disparate mistreatment [48]) requires classification algorithms to make predictions such that no group receives a disproportionately higher number of false-positive or false-negative errors.”
- Women also Snowboard: Overcoming Bias in Captioning Models (2018, background): “, age, gender, sexual orientation) in a fair manner is increasingly important as AI continues to become more ubiquitous [14,15,16,17].”
- A Step Toward More Inclusive People Annotations for Fairness (2021, background): “, equality of opportunity [13]) require partitioning an evaluation set by the attributes to which model performance should be agnostic.”
- Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy (2019, background): “Algorithmic solutions have ranged from removing undesired bias by preprocessing the data [39, 59], striking a tradeoff between performance and fairness by posing additional regularization during training or inference [34, 40, 52, 80, 81, 83], or designing application-specific interventions (such as ”
- Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints (2017, background): “Such issues have been demonstrated in various learning systems, including online advertisement systems (Sweeney, 2013), word embedding models (Bolukbasi et al., 2016; Caliskan et al., 2017), online news (Ross and Carter, 2011), web search (Kay et al., 2015), and credit score (Hardt et al., 2016).”
- Counterfactual Fairness in Text Classification through Robustness (2018, methodology): “h evaluate observational criteria, or statistical relationships between the data, group membership, the label, and the model’s prediction. Such metrics include demographic parity and equality of odds [5]. Hardt et al. [5] demonstrate that observational criteria such as equality of odds are insuffici”
- Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, background): “…to NLP; other fields in computer science such as data mining, machine learning, and security also study gender bias (Calders and Verwer, 2010; Feldman et al., 2015; Hardt et al., 2016; Misra et al., 2016; Kleinberg et al., 2016; Pleiss et al., 2017; Beutel et al., 2017; Kilbertus et al., 2017).”
- Towards A Rigorous Science of Interpretable Machine Learning (2017, background): “Some areas, such as the fairness [Hardt et al., 2016] and privacy [Toubiana et al., 2010, Dwork et al., 2012, Hardt and Talwar, 2010] the research communities have formalized their criteria, and these formalizations have allowed for a blossoming of rigorous research in these fields (without the need…”
- Mitigating Unwanted Biases with Adversarial Learning (2018, methodology): “Generality: The above method can be used to enforce DEMOGRAPHIC PARITY, EQUALITY OF ODDS, or EQUALITY OF OPPORTUNITY as described in Hardt et al. (2016). Further, it applies without modification to the cases when the output variable and/or protected variable are continuous instead of discrete.”
- Aligning AI With Shared Human Values (2020, background): “Research in algorithmic fairness initially began with simple statistical constraints (Lewis, 1978; Dwork et al., 2011; Hardt et al., 2016; Zafar et al., 2017), but these constraints were found to be mutually incompatible (Kleinberg et al.”
- Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting (2019, positive): “y computing the TPR gender gap— i.e., the difference in TPRs between genders—for each occupation. This notion of bias is closely related to the equality of opportunity fairness metric of Hardt et al. [25]. We choose to focus on TPR gender gaps because they enable us to study the ways in which gender”