Fix that Fix Commit: A real-world remediation analysis of JavaScript projects

By Vinuri Bandara, Thisura Rathnayake, Nipuna Weerasekara, Charitha Elvitigala, Kenneth M. Thilakarathna, Primal Wijesekera, C. Keppitiyagama. 2020

19 citations recorded by the source corpus.

While there is a large body of work on understanding vulnerabilities in the wild, little has been done to understand the dynamics of the remediation phase of the development cycle. To this end, we have done a timeline analysis on 118K commits from 53 of the most used JavaScript projects from GitHub to understand the provenance and prevalence of vulnerabilities in those projects. We used a vulnerability detector (CodeQL) to filter commits that introduced vulnerabilities and the commits that fixed a prior vulnerability. We found that in 82% of the projects, a commit fixing a prior vulnerability, in turn, introduced one or more new vulnerabilities. Among those projects, on average, 18% of the commits intended to fix vulnerabilities, in turn, introduced one or more new vulnerabilities. We also found that 50% of the total vulnerabilities found in those projects originated from a commit meant to fix a prior vulnerability, and 78% of those vulnerabilities could have been avoided if they were to use proper internal testing. We provide critical insights into how proper internal testing can avoid a significant portion of vulnerabilities, increasing organizations’ security posture.

Subfields: Information Systems, Software, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Ramesh Karri
2. Hammond A. Pearce
3. Brendan Dolan-Gavitt
4. Benjamin Tan
5. Baleegh Ahmad

In Software

1. Brendan Dolan-Gavitt
2. Benjamin Tan
3. Ramesh Karri
4. Baleegh Ahmad
5. Hammond A. Pearce

In Natural Language Processing

1. Brendan Dolan-Gavitt
2. Benjamin Tan
3. Ramesh Karri
4. Baleegh Ahmad
5. Hammond A. Pearce

In Signal Processing

1. Brendan Dolan-Gavitt
2. Ramesh Karri
3. Aleksander Madry

Citations in the corpus

• Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions (2021, methodology): “Prior work used CodeQL to identify vulnerable code commits in the life of a JavaScript project [22].”
• Examining Zero-Shot Vulnerability Repair with Large Language Models (2021, methodology): “Other prior work includes techniques and tools for detecting security bugs [19], [30], [49], [50].”
• An Empirical Cybersecurity Evaluation of GitHub Copilot's Code Contributions (2021, methodology): “Prior work used CodeQL to identify vulnerable code commits in the life of a JavaScript project [21].”