The Social Impact of Natural Language Processing

By Dirk Hovy, Shannon L. Spruit. 2016

418 citations recorded by the source corpus.

Medical sciences have long since established an ethics code for experiments, to minimize the risk of harm to subjects. Natural language processing (NLP) used to involve mostly anonymous corpora, with the goal of enriching linguistic analysis, and was therefore unlikely to raise ethical concerns. As NLP becomes increasingly wide-spread and uses more data from social media, however, the situation has changed: the outcome of NLP experiments and applications can now have a direct effect on individual users’ lives. Until now, the discourse on this topic in the ﬁeld has not followed the technological development, while public discourse was often focused on exaggerated dangers. This position paper tries to take back the initiative and start a discussion. We identify a number of social implications of NLP and discuss their ethical signiﬁcance, as well as ways to address them.

Subfields: Artificial Intelligence, Cultural Studies, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cultural Studies

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “A.2 Full categorization: Motivations Allocational harms Hovy and Spruit (2016); Caliskan et al. (2017); Madnani et al. (2017); Dixon et al. (2018); Kiritchenko and Mohammad (2018); Shen et al. (2018); Zhao et al. (2018b); Bhaskaran and Bhallamudi (2019); Bordia and Bowman (2019); Brunet et al.…”
Gender Bias in Coreference Resolution (2018, methodology): “Hovy and Spruit (2016) introduce a helpful set of terminology for identifying and categorizing types of bias that manifest in AI systems, including overgeneralization, which we observe in our work here.”
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, background): “Since people with disabilities are also more likely to talk about disability, this could impact their opportunity to participate equally in online fora (Hovy and Spruit, 2016), reducing their autonomy and dignity.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Increasing literature taxonomizes dangers [26], social impacts [110], sociotechnical harms [219], and social risks of all [80] or certain generative AI systems like language models [258], but evaluating these risks and impacts is a complementary yet distinct ongoing research area.”
The Capacity for Moral Self-Correction in Large Language Models (2023, background): “Scientifically, this may be useful as an additional experimental condition to test for misuse, as in [64], but practically there is much debate surrounding how to appropriately study dual-use issues arising from language models [22, 31].”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “…several years, the social impact of natural language processing and its applications has received increasing attention within the NLP community — see, for example, the overview by Hovy & Spruit (2016) — with Large Language Models (LLMs) as one of the recent primary targets (Bender et al., 2021).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “The social impact of natural language processing and its applications has received increasing attention within the NLP community (e.g. Hovy and Spruit, 2016) with Large Language Models (LLMs) as one of the recent primary targets (e.g. Bender et al., 2021; Bommasani et al., 2021; Weidinger et al.,…”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “AI systems can propagate harms such as exacerbating social inequity [64, 7, 41] and harmful biases [20, 13], which can be further amplified in larger systems as scale increases [6].”
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science (2018, background): “, 2017) and papers (Hovy and Spruit, 2016) in NLP, as well as workshops in related fields, notably the FATML series (http://www.”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, background): “See also the position paper by Hovy and Spruit (2016), which identiﬁes socio-ethical implications of the NLP systems in general.”
Social Bias in Elicited Natural Language Inferences (2017, background): “We focus on the problem of overgeneralization , in which a data mining model extrapolates excessively from observed patterns, leading to bias conﬁrmation among the model’s users (Hovy and Spruit, 2016).”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, background): “Besides, we refer the readers to Hovy and Spruit (2016) for a more general discussion of ethical concern in NLP.”
Demographic Dialectal Variation in Social Media: A Case Study of African-American English (2016, background): “This impacts many applications: for example, dialect speakers’ opinions may be mischaracterized under social media sentiment analysis or omitted altogether (Hovy and Spruit, 2016).”
Gender and Dialect Bias in YouTube’s Automatic Captions (2017, background): “Robust differences in accuracy of automatic speech recognition based on a speaker’s social identity is an ethical issue (Hovy and Spruit, 2016).”
Multi-Dimensional Gender Bias Classification (2020, background): “In particular, NLP models often learn biases against others based on their gender (Bolukbasi et al., 2016; Hovy and Spruit, 2016; Caliskan et al., 2017; Rudinger et al., 2017; Garg et al., 2018; Gonen and Goldberg, 2019; Dinan et al., 2019a).”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “Thus, data, tools, and inferences obtained for one purpose can be used for another purpose (Hovy and Spruit, 2016; Benton et al., 2017); yet, the risks associated with secondary uses may not be well understood.”