Attributing Fair Decisions with Attention Interventions

By Ninareh Mehrabi, Umang Gupta, Fred Morstatter, Greg Ver Steeg, Aram Galstyan. 2021

23 citations recorded by the source corpus.

The widespread use of Artificial Intelligence (AI) in consequential domains, such as health-care and parole decision-making systems, has drawn intense scrutiny on the fairness of these methods. However, ensuring fairness is often insufficient as the rationale for a contentious decision needs to be audited, understood, and defended. We propose that the attention mechanism can be used to ensure fair outcomes while simultaneously providing feature attributions to account for how a decision was made. Toward this goal, we design an attention-based model that can be leveraged as an attribution framework. It can identify features responsible for both performance and fairness of the model through attention interventions and attention weight manipulation. Using this attribution framework, we then design a post-processing bias mitigation strategy and compare it with a suite of baselines. We demonstrate the versatility of our approach by conducting experiments on two distinct data types, tabular and textual.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fred Morstatter
2. Ninareh Mehrabi
3. Aram Galstyan
4. Nripsuta Ani Saxena
5. Kristina Lerman
6. Ahmad Beirami
7. Fei Ye
8. Adrian G. Bors

In Safety Research

1. Aram Galstyan
2. Fred Morstatter
3. Ninareh Mehrabi
4. Kristina Lerman
5. Nripsuta Ani Saxena

In Safety, Robustness & Interpretability

1. Fred Morstatter
2. Ninareh Mehrabi
3. Aram Galstyan
4. Kristina Lerman
5. Ahmad Beirami
6. Nripsuta Ani Saxena

In AI Ethics, Fairness & Society

1. Aram Galstyan
2. Fred Morstatter
3. Ninareh Mehrabi
4. Kristina Lerman
5. Nripsuta Ani Saxena

Citations in the corpus

• A Survey on Bias and Fairness in Machine Learning (2019, background): “In [102], authors propose a post-processing bias mitigation strategy that utilizes attention mechanism for classification and that can provide interpretability.”
• Robust Conversational Agents against Imperceptible Toxicity Triggers (2022, methodology): “More recent work has considered attacks that target ethical concerns, such as triggering the models into outputting unfair predictions (Mehrabi et al., 2021b; Solans et al., 2021), or in the context of NLP, generating biased (Sheng et al., 2020) and toxic (Wallace et al., 2019) text.”