The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment

By Samiha Marwan, Nicholas Lytle, Joseph Jay Williams, Thomas William Price. 2019

45 citations recorded by the source corpus.

Automated hints, a powerful feature of many programming environments, have been shown to improve students' performance and learning. New methods for generating these hints use historical data, allowing them to scale easily to new classrooms and contexts. These scalable methods often generate next-step, code hints that suggest a single edit for the student to make to their code. However, while these code hints tell the student what to do, they do not explain why, which can make these hints hard to interpret and decrease students' trust in their helpfulness. In this work, we augmented code hints by adding adaptive, textual explanations in a block-based, novice programming environment. We evaluated their impact in two controlled studies with novice learners to investigate how our results generalize to different populations. We measured the impact of textual explanations on novices' programming performance. We also used quantitative analysis of log data, self-explanation prompts, and frequent feedback surveys to evaluate novices' understanding and perception of the hints throughout the learning process. Our results showed that novices perceived hints with explanations as significantly more relevant and interpretable than those without explanations, and were also better able to connect these hints to their code and the assignment. However, we found little difference in novices' performance. Our results suggest that explanations have the potential to make code hints more useful, but it is unclear whether this translates into better overall performance and learning.

Subfields: Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Human-Computer Interaction

Citations in the corpus

Comparing Code Explanations Created by Students and Large Language Models (2023, background): “Explanations help students develop their understanding of how a code snippet executes [27], which can help students improve their reasoning about writing their own code [29].”
Step Tutor: Supporting Students through Step-by-Step Example-Based Feedback (2020, background): “Although answering the prompt takes time, prior work suggests that self-explanation is critical for learning from feedback [5, 34, 44, 47], and such self-explanation does not emerge spontaneously without carefully-designed prompts [2, 18].”
Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book (2022, background): “Explanations help students to make connections and to develop their own understanding of how a code snippet executes [26].”