A Survey on Bias and Fairness in Machine Learning
By Ninareh Mehrabi, Fred Morstatter, Nripsuta Ani Saxena, Kristina Lerman, Aram Galstyan. ACM Computing Surveys, 2019
6,091 citations recorded by the source corpus.
With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.
Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability
View Paper: https://arxiv.org/abs/1908.09635
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Yoshua Bengio
- Jianfeng Gao
- Quoc V. Le
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
- Yejin Choi
In Safety Research
- Emilio Ferrara
- Daniel S. Schiff
- Kai-Wei Chang
- Michael Gira
- Ruisu Zhang
- Aram Galstyan
- Mark Coeckelbergh
- Laura Gustafson
In Safety, Robustness & Interpretability
- Kai-Wei Chang
- Nanyun Peng
- Nicolas Papernot
- Adina Williams
- Michael Backes
- Bo Li
- Olga Russakovsky
- Xingjun Ma
Citations in the corpus
- On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “Understanding these biases and the resulting ethical obligations have led to the rise of fair machine learning research [13, 15, 35].”
- BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (2021, background): “Recently, there has been growing evidence on how machine learning models without proper fairness checks risk reinforcing undesirable stereotypes, subjecting users to disparate treatment and enforcing de facto segregation [1, 23].”
- Does Gender Matter? Towards Fairness in Dialogue Systems (2019, background): “In other words, AI techniques may make decisions that are skewed towards certain groups of people in these applications (Mehrabi et al., 2019).”
- Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation (2021, background): “In general, it is important to be mindful of biases when using such machine learning systems (Torralba & Efros, 2011; Mehrabi et al., 2021).”
- Attributing Fair Decisions with Attention Interventions (2021, background): “The research in fairness concerns itself with various topics, such as defining fairness metrics, proposing solutions for bias mitigation, and analyzing existing harms in various systems (Mehrabi et al. 2021).”
- Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “” When measuring such harms, we consider the core concept of polarized perceptions: non-neutral views that can take the form of either prejudice that expresses negative views 3 or favoritism that expresses positive views toward a certain target perceived in the statement (Mehrabi et al., 2021).”
- Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition (2019, background): “From representation learning (Moyer et al. 2018) to graph embedding (Bose and Hamilton 2019) to community detection (Mehrabi et al. 2019a) and clustering (Backurs et al. 2019), researchers have studied biases in these areas and tried to address them by pointing out the observed problems and…”
- Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, background): “shows that the decisions made by machine learning models are sometimes biased and enforce inequality [69].”
- Learning to Prompt for Continual Learning (2021, background): “Different from class-incremental, task-incremental learning assumes task identity is known at test time and are often regarded as the simplest setting [34, 38].”
- Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “Classical fairness metrics [25, 29, 75] usually require specifying a protected attribute, such as sexual orientation or race.”
- Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020, background): “4 Related Work The fairness problems in natural language processing have received increasing attention (Mehrabi et al., 2019).”
- The Ghost in the Machine has an American accent: value conflict in GPT-3 (2022, background): “There has been extensive, and on-going, work on addressing the problem of biased word embeddings[29] in LLMs; however, the work tends to be focussed on specific pairings.”
- Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, neutral): “For an overview thereof, we refer the interested reader to respective surveys conducted by Zhang et al. (2019c) or Xu et al. (2019) for the first, and by Mehrabi et al. (2019) for the latter.”
- Advances, challenges and opportunities in creating data for trustworthy AI (2022, methodology): “Data design is critical for mitigating bias and ensuring the generalizability and reliability of the AI model trained on this data 25 .”
- Why So Toxic?: Measuring and Triggering Toxic Behavior in Open-Domain Chatbots (2022, background): “These tools are not perfect and, in some cases, biased [34, 52].”