Algorithms of oppression: how search engines reinforce racism

By Kinga Polynczuk-Alenius. 2019

2,756 citations recorded by the source corpus.

What can possibly be more mundane than a Google search? We use the search engine countless times day in day out, allowing it to classify, retrieve, rank and order the information we request. Trusti...

Subfields: Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “Algorithmic decisions, training data, and choices about how classes are defined and taxonomized (which we refer to informally as “class design”) can all contribute to and amplify social biases and inequalities resulting from the use of AI systems (Noble, 2018; Bechmann & Bowker, 2019; Bowker & Star,”
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “While previous studies have studied unintended biases in NLP models against other historically marginalized groups (Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2017; Barocas et al., 2017; Garg et al., 2019; Dixon et al., 2018; Noble, 2018; Manzini et al., 2019; Sap et al., 2019; May ”
Diagnosing Gender Bias in Image Recognition Systems (2020, background): “Because recent research suggests that GCV results may depend upon race and skin tone (Noble 2018), we control for race (coded as white or nonwhite).”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “The root causes of dataset bias are complex: they stem from bias in image search engines [52], data collection practices [74, 24], and real-world disparities.”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “AI systems can propagate harms such as exacerbating social inequity [64, 7, 41] and harmful biases [20, 13], which can be further amplified in larger systems as scale increases [6].”
Large image datasets: A pyrrhic win for computer vision? (2020, background): “Image classiﬁcation and labelling practices, without the necessary precautions and awareness of these problematic histories, pick up these stereotypes and prejudices and perpetuate them [31, 58, 59].”
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets (2020, background): “Sociotechnical systems abound in ways that they have failed people of color (Noble, 2018; Benjamin, 2019), women (Bolukbasi et al., 2016), LGBTQ+ communities (Scheuerman et al., 2019), people with disabilities (Hutchinson et al., 2020; Trewin, 2018), and the working class and those in poverty…”
The Values Encoded in Machine Learning Research (2021, background): “Others [10, 19, 50] examine technology and data science from an anti-racist and intersectional feminist lens, discussing how our infrastructure has largely been built by and for white men; D’Ignazio and Klein [19] present a set of alternative principles and methodologies for an intersectional femini”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, neutral): “, 2019), sociolinguistics (Labov, 1986; Cheshire, 2007), and critical race theory (Noble, 2018; Benjamin, 2019).”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, background): “Yet, there is a growing abundance of examples of how these systems are failing people of color (Noble, 2018; Benjamin, 2019), women (Bolukbasi et al.”
Annotating Twitter Data from Vulnerable Populations: Evaluating Disagreement Between Domain Experts and Graduate Student Annotators (2018, background): “In addition, Safiya Noble [11] warns us that biases unchecked in the labeling of images on google reinforce racist stereotypes.”
Better, Nicer, Clearer, Fairer: A Critical Assessment of the Movement for Ethical Artificial Intelligence and Machine Learning (2019, background): “racism corrupt what could be considered public information resources [66], [67].”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “When an end user is presented with information online that stigmatizes them based on their race, age, or gender, or doesn’t accurately perceive their identity, it causes harm [34, 36, 37, 41].”
Equity Beyond Bias in Language Technologies for Education (2019, background): “This may take the form of search results returning stereotypical images of minorities (Noble, 2018) or other algorithmic stereotyping (Abbasi et al., 2019); much of the work in word embeddings falls into this category (Caliskan et al., 2017), though research on downstream tasks and outcomes often…”
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, background): “Entire under-represented groups can show up as edge cases, with profound social implications [88].”
The Roles of Personality Traits, AI Anxiety, and Demographic Factors in Attitudes toward Artificial Intelligence (2022, background): “Frequently highlighted ethical issues include racism arising from AI-powered decision systems (Lyons, 2020; Sanjay, 2020), potential data privacy violations that some have suggested have been carried out by some large technology companies, e.g., tapping clients using AI technologies (Nicolas,…”