The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development
By Steven I. Ross, Fernando Martinez, Stephanie Houde, Michael J. Muller, Justin D. Weisz. International Conference on Intelligent User Interfaces, 2023
335 citations recorded by the source corpus.
Large language models (LLMs) have recently been applied in software engineering to perform tasks such as translating code between programming languages, generating code from natural language, and autocompleting code as it is being written. When used within development tools, these systems typically treat each model invocation independently from all previous invocations, and only a specific limited functionality is exposed within the user interface. This approach to user interaction misses an opportunity for users to more deeply engage with the model by having the context of their previous interactions, as well as the context of their code, inform the model’s responses. We developed a prototype system – the Programmer’s Assistant – in order to explore the utility of conversational interactions grounded in code, as well as software engineers’ receptiveness to the idea of conversing with, rather than invoking, a code-fluent LLM. Through an evaluation with 42 participants with varied levels of programming experience, we found that our system was capable of conducting extended, multi-turn discussions, and that it enabled additional knowledge and capabilities beyond code generation to emerge from the LLM. Despite skeptical initial expectations for conversational programming assistance, participants were impressed by the breadth of the assistant’s capabilities, the quality of its responses, and its potential for improving their productivity. Our work demonstrates the unique potential of conversational interactions with LLMs for co-creative processes like software development.
Subfields: Artificial Intelligence, Information Systems
View Paper: https://www.semanticscholar.org/paper/f5d93cf5d81575aeee61faeddde4437fbcb74cd0
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Information Systems
- Shin Yoo
- Mark Harman
- Shubho Sengupta
- Mitya Lyubarskiy
- Jie M. Zhang
- Angela Fan
- Beliz Gokkaya
- Phil McMinn
Citations in the corpus
- Large Language Models for Software Engineering: Survey and Open Problems (2023, background): “For example, Ross et al. [208], introduced an LLM-based programmers’ assistant, evaluating its deployment with 42 participants while Tian et al. [209] highlighted the attention span limitations of ChatGPT.”
- Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes (2024, background): “One of the most common uses of LLMs within software engineering is code generation (e.g., [12, 24, 31, 42, 45, 54, 82, 91, 93, 96, 121, 128]).”