Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

By Gustavo Sandoval, Hammond A. Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, Brendan Dolan-Gavitt. 2022

184 citations recorded by the source corpus.

Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers' code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked 'shopping list' structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.

Subfields: Health Informatics, Information Systems, Large Language Models, Code Generation & Program Synthesis, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Large Language Models

In Code Generation & Program Synthesis

In Privacy & Security

Citations in the corpus

StarCoder: may the source be with you! (2023, background): “However, the security implications should also be carefully considered (Sandoval et al., 2023).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Sandoval et al. [225] conducted a user study (58 users) to assess the security implications of LLMs, particularly OpenAI Codex, as code assistants for developers.”
VeriGen: A Large Language Model for Verilog Code Generation (2023, background): “security [18, 24]), there is scant examination of LLMs for hardware applications.”
The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023, methodology): “Some focused on finding interaction patterns [32, 148, 176] while others focused more on how productivity is impacted by the use of models [101, 131], whether code produced when using AI code generators is less secure than when not [162], and how students use code explanations generated by”