Ethical and social risks of harm from Language Models

By Laura Weidinger, John F. J. Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh and 13 more. 2021

1,580 citations recorded by the source corpus.

This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understanding of the potential risks posed by these models is needed. A wide range of established and anticipated risks are analysed in detail, drawing on multidisciplinary expertise and literature from computer science, linguistics, and social sciences. We outline six specific risk areas: I. Discrimination, Exclusion and Toxicity, II. Information Hazards, III. Misinformation Harms, V. Malicious Uses, V. Human-Computer Interaction Harms, VI. Automation, Access, and Environmental Harms. The first area concerns the perpetuation of stereotypes, unfair discrimination, exclusionary norms, toxic language, and lower performance by social group for LMs. The second focuses on risks from private data leaks or LMs correctly inferring sensitive information. The third addresses risks arising from poor, false or misleading information including in sensitive domains, and knock-on risks such as the erosion of trust in shared information. The fourth considers risks from actors who try to use LMs to cause harm. The fifth focuses on risks specific to LLMs used to underpin conversational agents that interact with human users, including unsafe use, manipulation or deception. The sixth discusses the risk of environmental harm, job automation, and other challenges that may have a disparate effect on different social groups or communities. In total, we review 21 risks in-depth. We discuss the points of origin of different risks and point to potential mitigation approaches. Lastly, we discuss organisational responsibilities in implementing mitigations, and the role of collaboration and participation. We highlight directions for further research, particularly on expanding the toolkit for assessing and evaluating the outlined risks in LMs.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

Training Compute-Optimal Large Language Models (2022, neutral): “Large Language Models carry potential risks such as outputting oﬀensive language, propagating social biases, and leaking private information (Bender et al., 2021; Weidinger et al., 2021).”
PaLM: Scaling Language Modeling with Pathways (2022, background): “…al., 2021; Bommasani & et. al., 2021), for instance, perpetuating or exacerbating social stereotypes and disparities that are reflected in training data (Sheng et al., 2021), memorizing and revealing private information (Carlini et al., 2022), or causing downstream harms (Weidinger et al., 2021).”
Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “Bender et al. (2021b) and Weidinger et al. (2021) underscore various hazards like bias, toxicity, private data leakage, and the potential for malicious uses.”
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, background): “As has been shown in other work (for example [9, 71, 53]), bias issues that exist in the dataset can be learned by models as they are trained on the data.”
LaMDA: Language Models for Dialog Applications (2022, background): “More research is needed to develop robust standards for safety and fairness that capture the many dimensions of risk [54] in general-purpose dialog models such as LaMDA.”
Competition-level code generation with AlphaCode (2022, background): “Tools that improve the productivity of developers would also improve the productivity of developers writing malicious code (Chen et al., 2021; Weidinger et al., 2021).”
Emergent Abilities of Large Language Models (2022, neutral): “In §5.3, we saw how emergent performance on MMLU for Gopher and Chinchilla could be viewed as a function of training FLOPs, model parameters, and WikiText103 perplexity.”
OpenAssistant Conversations - Democratizing Large Language Model Alignment (2023, background): “We recognize that sufﬁciently powerful language models can have a signiﬁcant impact on society [35], and therefore we believe it is essential to promote transparency in their development and deployment.”
The Capacity for Moral Self-Correction in Large Language Models (2023, background): “Large language models exhibit harmful social biases [1, 6, 8, 11, 15, 24, 29, 50, 62] that can sometimes get worse for larger models [2, 18, 20, 43, 55].”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, methodology): “To develop the list of topic tags, we referred to the taxonomies of potential harms of language models in [48, 57], industry content moderation guidelines, and a manual review of the top 100 most harmful conversations in our dataset.”
Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “Misinformation rules are drawn from Section 2.3 of Weidinger et al. (2021) (Misinformation harms).”
Aligning Language Models with Preferences through f-divergence Minimization (2023, background): “Distributional preferences Finally, there is a class of distributional preferences (Weidinger et al., 2021) that cannot be expressed as a function of a single sample x but depend on the entire distribution, e.g. a particular gender distribution of persons mentioned in LM samples.”
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting (2022, background): “…and molecular structures (Jaques et al., 2017a), code compilability (Korbak et al., 2021), reducing distributional biases (Khalifa et al., 2021; Weidinger et al., 2021), evaluation metrics for Machine Translation and Summarization (Ranzato et al., 2016; Bahdanau et al., 2016), or direct human…”