An Empirical Cybersecurity Evaluation of GitHub Copilot's Code Contributions
By Hammond A. Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri. 2021
51 citations recorded by the source corpus.
Subfields: Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Safety, Robustness & Interpretability
- Jacob Steinhardt
- Nicholas Carlini
- Dan Hendrycks
- Pushmeet Kohli
- Sven Gowal
- Matthias Hein
- Xinyun Chen
- Nate Kushman
Citations in the corpus
- PaLM: Scaling Language Modeling with Pathways (2022, background): “, 2018) and security (Pearce et al., 2021) but that line of research is still early and requires further work as code models evolve.”
- Competition-level code generation with AlphaCode (2022, background): “Models can generate code with exploitable weaknesses, either unintentional vulnerabilities from outdated code or intentional ones injected by malicious actors into the training set (Pearce et al., 2021).”
- Execution-Based Evaluation for Open-Domain Code Generation (2022, background): “We should always keep alert to concealing malicious code (Wallace et al., 2021) or generating code with security vulnerabilities (Verdi et al., 2020; Pearce et al., 2021).”
- A Hazard Analysis Framework for Code Synthesis Large Language Models (2022, positive): “A concurrent study has demonstrated these results further using Github’s CodeQL [31].”
- Unsolved Problems in ML Safety (2021, background): “[136] Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri.”
- CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning (2022, background): “For example, pretraining data from public Github code repositories may contain vulnerabilities and the resulting synthesis models may generate programs with weak security measures [Hammond Pearce et al., 2021].”
- MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages (2022, background): “Nevertheless, with the advances in models capable of generating code from natural language inputs, we should be aware of the potentially harmful usage such as concealing malicious code (Wallace et al., 2020), or generating code with security vulnerabilities (Verdi et al., 2020; Pearce et al., 2021).”
- Jigsaw: Large Language Models meet Program Synthesis (2021, background): “Ideally, the synthesized code should have high performance, should not have security vulnerabilities [29], and respect licensing attribution 4.”