Release Strategies and the Social Impacts of Language Models
By Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Jasmine Wang. 2019
880 citations recorded by the source corpus.
Large language models have a range of beneficial uses: they can assist in prose, poetry, and programming; analyze dataset biases; and more. However, their flexibility and generative capabilities also raise misuse concerns. This report discusses OpenAI's work related to the release of its GPT-2 language model. It discusses staged release, which allows time between model releases to conduct risk and benefit analyses as model sizes increased. It also discusses ongoing partnership-based research and provides recommendations for better coordination and responsible publication in AI.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Yejin Choi
- Noah A. Smith
- Yoshua Bengio
- Jason Aaron Edward Weston
In Safety, Robustness & Interpretability
- Eric Michael Smith
- Adina Williams
- Kai-Wei Chang
- Dirk Hovy
- Percy Liang
- Y-Lan Boureau
- Nanyun Peng
- Maarten Sap
In AI Ethics, Fairness & Society
- Adrian Weller
- Zeerak Talat
- William Yang Wang
- Emily M. Bender
- Alexandra Sasha Luccioni
- Gavin Abercrombie
- Verena Rieser
- Margaret Mitchell
Citations in the corpus
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, background): “Since RAG can be employed as a language model, similar concerns as for GPT-2 [50] are valid here, although arguably to a lesser extent, including that it might be used to generate abuse, faked or misleading content in the news or on social media; to impersonate others; or to automate the production ”
- Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “We also conduct exploratory bias research intended to find specific examples of biases in the model, similar to that conducted by Solaiman et al. (2019).”
- Training language models to follow instructions with human feedback (2022, background): “…2021; Liang et al., 2021; Manela et al., 2021; Caliskan et al., 2017; Kirk et al., 2021), leak private data (Carlini et al., 2021), generate misinformation (Solaiman et al., 2019; Buchanan et al., 2021), and be used maliciously; for a thorough review we direct the reader to Weidinger et al. (2021).”
- Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, methodology): “We observe that in the majority of cases, the GPT-2 model exhibits larger (i.e. worse) I.F. and G.F. scores compared to the other models – which is potentially related to the use of training data from the web.”
- Automatic Detection of Generated Text is Easiest when Humans are Fooled (2019, background): “Total Probability Solaiman et al. (2019) pro-pose a very simple baseline consisting of a threshold on the total probability of the text sequence.”
- Ethical and social risks of harm from Language Models (2021, background): “Responsible publication norms and considerations of accessibility are also key, as they determine who can develop LM use cases or applications (Solaiman et al., 2019).”
- Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “…2019) There are also metrics for other bias evaluation setups in continuation generation tasks involving sentiment (Shwartz et al., 2020), the ratio of gendered words (Solaiman et al., 2019; Vig et al., 2020; Dinan et al., 2020a), and other novel metrics (Peng et al., 2020; Yeo and Chen, 2020).”
- Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “In the broader ecosystem, methods of deployment [229] affect social impact.”
- The Curse of Recursion: Training on Generated Data Makes Models Forget (2023, background): “These models now (arguably) pass a weaker form of the Turing test in the sense that their output cannot be reliably distinguished from text written by humans [Solaiman et al., 2019].”
- Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “If multiple model versions exist which might be expected to have differing impacts, the researchers might consider adopting a staged release policy, as in Solaiman et al. (2019).”
- Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “In addition, work on building safer LLMs explores fine-tuning on curated data (Solaimon and Dennison, 2021) or directly controlling the generations of the model (Dathathri et al., 2019; Liu et al., 2021; Schick et al., 2021; Xu et al., 2020).”
- BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “Following Solaiman et al. (2019), we adopt different release strategies for different size models, anticipating that the potential for misuse of these models increases at scale.”
- What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? (2022, methodology): “Across all experiments undertaken over the course of this study (including unreported preliminary and failed experiments), we performed training for 1,854 hours on 64 chips (TPUv4-128) and 1,395 hours on 512 TPUv4 chips (TPUv4-1024) for a total of 832,896 chip-hours.”
- GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (2023, neutral): “(Khlaaf et al., 2022; Weidinger et al., 2022; Solaiman et al., 2019)”
- The Gradient of Generative AI Release: Methods and Considerations (2023, background): “In 2019, OpenAI stage-released language model GPT-2 in four sizes by increasing parameter count over nine months while conducting research internally and with external partners [91].”
- Unified Pre-training for Program Understanding and Generation (2021, methodology): “Later, BERT’s task-independent pre-training approach is rigorously studied (Devlin et al., 2019; Liu et al., 2019; Solaiman et al., 2019; Feng et al., 2020; Sun et al., 2019; Li et al., 2020).”
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “CodeGPT has the same model architecture and training objectives of GPT-2 [53], which consists of 12 layers of Transformer decoders.”
- The Radicalization Risks of GPT-3 and Advanced Neural Language Models (2020, positive): “[1] Testing across white supremacy, Islamism, anarchism, and Marxist-Leninism, CTEC’s analysis demonstrated that GPT-2 represented the frontier of a novel threat to the information landscape: text generation at a speed, scale, and ease of use that exceeds that of troll farms and human-run…”