Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer

By David Madras, Toniann Pitassi, Richard S. Zemel. 2017

324 citations recorded by the source corpus.

In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and an external decision-maker. The model can choose to say PASS, and pass the decision downstream, as explored in rejection learning. We extend this concept by proposing "learning to defer", which generalizes rejection learning by considering the effect of other agents in the decision-making process. We propose a learning algorithm which accounts for potential biases held by external decision-makers in a system. Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system.

Subfields: Cognitive Neuroscience, Economics and Econometrics, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Economics and Econometrics

In Safety Research

Citations in the corpus

Organizational Decision-Making Structures in the Age of Artificial Intelligence (2019, background): “45 Human decisions as input to algorithmic decision making.”
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, background): “Uncertainty in predictions can also be used to abstain from making decisions or defer the decisions to experts, which can lead to overall improvement in accuracy and fairness of the predictions [120].”
Making Decisions that Reduce Discriminatory Impacts (2019, unknown): “The second work is by Madras et al. (2018) who consider a model where some algorithmic predictions can be deferred to a black-box decision maker (e.g., a human, proprietary software).”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “As recent work has suggested [4, 55, 59, 79], it may be useful to directly optimize for complementary behavior by accounting for the human behavior during training, who may have access to a diferent set of features [72].”
Fairness in Machine Learning (2020, methodology): “…59, 97, 135, 183, 185] or add penalties to the objective [5, 14, 22, 44, 46, 56, 58, 62, 73, 74, 75, 79, 82, 87, 88, 89, 93, 94, 95, 98, 104, 115, 119, 123, 133, 134, 135, 138, 139, 154, 165, 176, 179, 180, 182], others use adversary techniques to maximize the system ability to predict the…”
Consistent Estimators for Learning to Defer to an Expert (2020, methodology): “Note that the expert deferral setting considered here can be thought of as a hard mixture of two experts problem where one of the experts is fixed (Jordan & Jacobs, 1994; Shazeer et al., 2017; Madras et al., 2018).”