AI can be sexist and racist — it’s time to make it fair

By James Y. Zou, Londa Schiebinger. 2018

641 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor (2019, background): “…Mamidi 2017; Gebru et al. 2018; Mohammad et al. 2018; Hall Maudslay et al. 2019), also outside NLP and artiﬁcial intelligence (McQuillan 2018) and in popularized articles (Zou and Schiebinger 2018), where readers are usually in no position to verify the reliability or signiﬁcance of such examples.”
Ethical and social risks of harm from Language Models (2021, background): “…applications ranging from diagnostic healthcare algorithms (Obermeyer et al., 2019) to social outcome prediction (Narayanan, 2021); for a more general introduction see (Chouldechova and Roth, 2018; Kordzadeh and Ghasemaghaei, 2021; Mehrabi et al., 2021; Noble, 2018; Zou and Schiebinger, 2018).”
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, background): “The tendency of machine learning models to produce worse outcomes for groups which are underrepresented in their training data (in terms of race, gender, or otherwise) is well documented [63].”
A Survey on Bias and Fairness in Machine Learning (2019, background): “his case that we can clearly observe the bias towards dark-skin females, as previously dark-skin males would compromise for dark-skin females and would hide the underlying bias towards this subgroup. [142] also analyzes some examples of the biases that can exist in the data and algorithms and offer ”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation (2021, background): “AI and machine learning communities have increasingly paid attention to algorithmic fairness and dataset and model biases [72, 11, 77, 73].”
Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of Medical Images: Results from Recently Published Papers (2019, background): “Therefore, individuals developing such AI algorithms might rely on whatever data are available (methodologically referred to as convenience case-control data), although these may be prone to selection biases and artificial disease prevalence and likely not represent real-world clinical settings well”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey (2020, background): “This could generate a robust AI model [81].”
Hurtful words: quantifying biases in clinical contextual word embeddings (2020, background): “…exist within both traditional non-contextual word embeddings [9, 73] and contextual word embeddings [36, 62, 72], biases in pretrained contextual embedding models can additionally encode historical biases in the training corpora, class imbalance in datasets, and data quality differences [53, 74].”
Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review (2021, neutral): “Finally, the issue of fairness arises with algorithm bias when data used to train and test chatbots do not accurately reflect the people they represent [101].”
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets (2021, background): “Machine learning methods trained on biased data may perpetuate these biases in undesirable ways [31, 60].”
Advances, challenges and opportunities in creating data for trustworthy AI (2022, background): “Such coverage gaps introduce bias and limit the generalizability of the AI models to diverse populations 28,29 .”