Using an LLM to Help with Code Understanding

By Daye Nam, A. Macvean, Vincent J. Hellendoorn, Bogdan Vasilescu, Brad A. Myers. International Conference on Software Engineering, 2023

626 citations recorded by the source corpus.

Understanding code is challenging, especially when working in new and complex development environments. Code comments and documentation can help, but are typically scarce or hard to navigate. Large language models (LLMs) are revolutionizing the process of writing code. Can they do the same for helping understand it? In this study, we provide a first investigation of an LLM-based conversational UI built directly in the IDE that is geared towards code understanding. Our IDE plugin queries OpenAI's GPT-3.5-turbo model with four high-level requests without the user having to write explicit prompts: to explain a highlighted section of code, provide details of API calls used in the code, explain key domainspecific terms, and provide usage examples for an API. The plugin also allows for open-ended prompts, which are automatically contextualized to the LLM with the program being edited. We evaluate this system in a user study with 32 participants, which confirms that using our plugin can aid task completion more than web search. We additionally provide a thorough analysis of the ways developers use, and perceive the usefulness of, our system, among others finding that the usage and benefits differ between students and professionals. We conclude that in-IDE prompt-less interaction with LLMs is a promising future direction for tool builders.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/f5ea85f67fc5ea9cce2c5345ed92740f7377cb2e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Juho Leinonen
2. Andrew Luxton-Reilly
3. James Prather
4. Hieke Keuning
5. Paul Denny
6. Sami Sarsa
7. Arto Hellas

In Software

1. Andrew Luxton-Reilly
2. Hieke Keuning
3. Juho Leinonen

Citations in the corpus

• The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023, background): “Some focused on finding interaction patterns [32, 148, 176] while others focused more on how productivity is impacted by the use of models [101, 131], whether code produced when using AI code generators is less secure than when not [162], and how students use code explanations generated by”
• Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes (2024, unknown): “LLMs can explain and summarize code [45, 81, 82, 128].”