Active Fairness in Algorithmic Decision Making

By Alejandro Noriega-Campero, Michiel A. Bakker, Bernardo Garcia-Bulle, Alex Pentland. 2018

90 citations recorded by the source corpus.

Society increasingly relies on machine learning models for automated decision making. Yet, efficiency gains from automation have come paired with concern for algorithmic discrimination that can systematize inequality. Recent work has proposed optimal post-processing methods that randomize classification decisions for a fraction of individuals, in order to achieve fairness measures related to parity in errors and calibration. These methods, however, have raised concern due to the information inefficiency, intra-group unfairness, and Pareto sub-optimality they entail. The present work proposes an alternativeactive framework for fair classification, where, in deployment, a decision-maker adaptively acquires information according to the needs of different groups or individuals, towards balancing disparities in classification performance. We propose two such methods, where information collection is adapted to group- and individual-level needs respectively. We show on real-world datasets that these can achieve: 1) calibration and single error parity (e.g.,equal opportunity ); and 2) parity in both false positive and false negative rates (i.e.,equal odds ). Moreover, we show that by leveraging their additional degree of freedom,active approaches can substantially outperform randomization-based classifiers previously considered optimal, while avoiding limitations such as intra-group unfairness.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Algorithmic Fairness (2020, methodology): “This dataset was studied, for example, in [106].”
Fairness in Machine Learning: A Survey (2020, background): “Calibration is particularly useful when the output is not a direct decision but used to inform human judgment when assessing risks (e.g., awarding a loan) [223].”
Fairness in Machine Learning (2020, methodology): “Post-Processing methods consist in transforming the model outputs in order to make them fair [2, 7, 31, 42, 51, 54, 68, 69, 99, 100, 106, 137, 147, 153, 156, 178].”
Adaptive Sampling Strategies to Construct Equitable Training Datasets (2022, background): “underserved groups, such as screening [6, 14, 53], and designing more equitable interventions given a set of risk scores [18].”
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, background): “Algorithmic fairness approaches that employ active learning to either acquire features [137] or samples [4] with accuracy and fairness objectives also fall under this category.”
A Review on Fairness in Machine Learning (2022, methodology): “This dataset was studied, for example, in the work of Bakker et al. [11] and Noriega-Campero [157].”