Re-imagining Algorithmic Fairness in India and Beyond

By Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, Vinodkumar Prabhakaran. 2021

269 citations recorded by the source corpus.

Conventional algorithmic fairness is West-centric, as seen in its subgroups, values, and methods. In this paper, we de-center algorithmic fairness and analyse AI power in India. Based on 36 qualitative interviews and a discourse analysis of algorithmic deployments in India, we find that several assumptions of algorithmic fairness are challenged. We find that in India, data is not always reliable due to socio-economic factors, ML makers appear to follow double standards, and AI evokes unquestioning aspiration. We contend that localising model fairness alone can be window dressing in India, where the distance between models and oppressed communities is large. Instead, we re-imagine algorithmic fairness in India and provide a roadmap to re-contextualise data and models, empower oppressed communities, and enable Fair-ML ecosystems.

Subfields: Safety Research, Sociology and Political Science, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “, 2020), or biases that are salient in non-Western socio-cultural contexts, such as caste (Sambasivan et al., 2021).”
LaMDA: Language Models for Dialog Applications (2022, background): “For example, bias mitigations in certain contexts might have counter-intuitive impacts in other geocultural contexts [101].”
Ethical and social risks of harm from Language Models (2021, background): “For example, the Hindu caste system underpins discrimination in India, but not across the globe (Sambasivan et al., 2021).”
Societal Biases in Language Generation: Progress and Challenges (2021, background): “Bias Evaluations It is difﬁcult to evaluate societal biases in NLG tasks because NLG can be open-domain, and there are many different notions of biases from various backgrounds and cultures (Sambasivan et al., 2021).”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Moverover, as geographic and cultural contexts shift, so do the meaning of different categories [206, 112] and must be interpreted according to their local meaning.”
Re-contextualizing Fairness in NLP: The Case of India (2022, methodology): “However, the majority of this research is framed in and for the Western context, relying on data and values reflecting the West (Sambasivan et al., 2021).”
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, background): “In particular, [47] points out that the technical, social, ethical, and physical distance between the builders of a technology and the communities it is meant to serve is large, in such settings.”
PaLM 2 Technical Report (2023, methodology): “(Weidinger et al., 2021; Shelby et al., 2023; Sambasivan et al., 2021), and additionally evaluate memorization as a proxy for potential privacy harms (Ramesh et al.”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “Sambasivan et al. (2021) and Chan et al. (2021) contend there remains a signiﬁcant gap between the communities governing and governed by AI, and advocate for a redistribution of powers and responsibilities in developing responsible AI.”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “…challenging to create because existing measurement tools, like toxicity classi-ﬁers, do not work well in all languages [69], cultural norms might not transfer [88], assumptions in benchmark design might not translate, 8 and there may be fewer qualiﬁed native speakers on common annotation platforms.”
Taxonomy of Risks posed by Language Models (2022, background): “The axes along which unfair bias is encoded in the LM can be rooted in localised social hierarchies such as the Hindu caste system, making it harder to anticipate harmful social stereotypes across contexts [163].”
Evaluation Gaps in Machine Learning Practice (2022, background): “Also overlooked are inference-time externalities of energy use [28, 60], cultural aspects of the ecosystem [145], and long term impacts [29].”
Cultural Incongruencies in Artificial Intelligence (2022, background): “Fairness evaluations framed in the West may not readily apply to other socio-cultural contexts [23].”
How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India (2022, background): “Unfortunately, these preconditions may be unique to Global North contexts [109].”
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, background): “, [31, 98, 107], understanding socio-cultural factors e.”
Changing the World by Changing the Data (2021, background): “…sub-groups, their values, and underlying social constructs may also differ across communities: e.g. both in India and US there is discrimination based on skin tone, but in the US context it stands for race, and in India it is a proxy for ethnicity, caste and class (Sambasivan et al., 2021a).”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “proliferating false ideas about cultural groups [78, 189].”
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media (2021, background): “However, this reliance on lexicons and annotated data limit the focus to a handful of social injustices for which resources/lexicons are available, and largely ignores biases in non-Western contexts (Sambasivan et al., 2021) and online environments containing creative or noisy language variation.”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “162) [164], the nature of their harm can encompass adverse cultural such as systemic erasure [61], eurocentric ideas being exported to Global South [67, 129], harmful cultural beliefs [66], such as normalizing a culture of non-consensual sexual activity [123], or proliferating false ideas about cult”
Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, unknown): “Prior work also points out that development of AI systems is predominantly centered around Western contexts (Sambasivan et al., 2021; Prabhakaran et al., 2022).”