A Hazard Analysis Framework for Code Synthesis Large Language Models

By Heidy Khlaaf, Pamela Mishkin, Josh Achiam, Gretchen Krueger, Miles Brundage. 2022

40 citations recorded by the source corpus.

Codex, a large language model (LLM) trained on a variety of codebases, exceeds the previous state of the art in its capacity to synthesize and generate code. Although Codex provides a plethora of benefits, models that may generate code on such scale have significant limitations, alignment problems, the potential to be misused, and the possibility to increase the rate of progress in technical fields that may themselves have destabilizing impacts or have misuse potential. Yet such safety impacts are not yet known or remain to be explored. In this paper, we outline a hazard analysis framework constructed at OpenAI to uncover hazards or safety risks that the deployment of models like Codex may impose technically, socially, politically, and economically. The analysis is informed by a novel evaluation framework that determines the capacity of advanced code generation techniques against the complexity and expressivity of specification prompts, and their capability to understand and execute them relative to human ability.

Subfields: Computer Networks and Communications, Information Systems, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Code generative systems can be evaluated for their limitations [56] and hazards [127], from alignment questions like producing bugs and harmful biases, to economic and environmental impacts (see Section 4.”
StarCoder: may the source be with you! (2023, background): “We refer the reader to Chen et al. (2021, Section 7) for a broader impact analysis of Code LLMs, as well as Khlaaf et al. (2022) for an in-depth risk assessment and hazard analysis of this emerging technology.”
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (2023, neutral): “(Khlaaf et al., 2022; Weidinger et al., 2022; Solaiman et al., 2019)”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Threat modeling will necessarily differ by modality, but as systems improve in types of outputs such as code generation, potential harms can also broaden [45].”
The Stack: 3 TB of permissively licensed source code (2022, background): “, 2022) or unsafe programs (Khlaaf et al., 2022) during deployment.”
Programming Is Hard - Or at Least It Used to Be: Educational Opportunities and Challenges of AI Code Generation (2022, background): “, economic [7], political [18]) considerations presented by AI-generated code, nor those specific to advanced/professional programmers (e.”