sk_p: a neural program corrector for MOOCs

By Yewen Pu, Karthik Narasimhan, Armando Solar-Lezama, Regina Barzilay. 2016

128 citations recorded by the source corpus.

We present a novel technique for automatic program correction in MOOCs, capable of fixing both syntactic and semantic errors without manual, problem specific correction strategies. Given an incorrect student program, it generates candidate programs from a distribution of likely corrections, and checks each candidate for correctness against a test suite. The key observation is that in MOOCs many programs share similar code fragments, and the seq2seq neural network model, used in the natural-language processing task of machine translation, can be modified and trained to recover these fragments. Experiment shows our scheme can correct 29% of all incorrect submissions and out-performs state of the art approach which requires manual, problem specific correction strategies.

Subfields: Artificial Intelligence, Information Systems, Software, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Software

In Code Generation & Program Synthesis

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, methodology): “Previous works in code repair apply random or heuristic perturbations to good code (e.g., dropping tokens) and prepare synthetic paired data 〈perturbed code, good code〉 to train a fixer (Pu et al., 2016; Gupta et al., 2017; Ahmed et al., 2018; Hajipour et al., 2019; Yasunaga & Liang, 2020).”
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback (2020, background): “There is increasing interest in automatic correction of introductory programming assignments (Pu et al., 2016; Parihar et al., 2017; Ahmed et al., 2018).”
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair (2018, background): “sk_p is a program repair technique for syntactic and semantic errors in student programs submitted to MOOCs [23].”
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation (2018, background): “To support programmers in this common activity, researchers have proposed a number of approaches aimed at automatically repairing programs [5, 8, 20, 21, 36, 37, 41, 42, 45, 48, 52, 53, 60, 63, 65, 75, 77, 83, 85, 88].”
Neural Program Repair by Jointly Learning to Localize and Repair (2019, methodology): “Sk p (Pu et al., 2016) is another enumerative neural program repair approach to repair student programs using an encoder-decoder architecture.”
Accelerating search-based program synthesis using learned probabilistic models (2018, background): “As a result, recent years have witnessed a surge of interest in applying this technology to a wide range of problems, including end-user programming [12], intelligent tutoring [27], circuit transformation [9], and program repair [20], among many others.”
Neural Attribution for Semantic Bug-Localization in Student Programs (2019, background): “[21] propose a deep learning based technique for both syntactic and semantic error repair in small student programs.”
Neuro-Symbolic Program Corrector for Introductory Programming Assignments (2018, methodology): “Sk _p [22] uses a skip-gram neural network model to predict a program statement using the lines before and after the erroneous line.”
Dynamic Neural Program Embedding for Program Repair (2017, methodology): “DeepFix (Gupta et al., 2017), SynFix (Bhatia & Singh, 2016), and sk p (Pu et al., 2016) are recent neural program repair techniques for correcting errors in student programs for MOOC assignments, and they all represent programs as sequences of tokens.”
Compilation Error Repair: For the Student Programs, From the Student Programs (2018, background): “[21] performs a brute force replacement, starting from the first-statement to the last; relying on their model to regenerate the correct statements, while fixing incorrect ones.”
Pre-trained Contextual Embedding of Source Code (2019, background): “, 2016), program repair (Pu et al., 2016; Gupta et al., 2017), and natural language to code mapping (Iyer et al.”
Learning semantic program embeddings with graph interval neural network (2020, methodology): “Earlier works predominately considered source code as text, and mechanically transferred natural language methods to discover their shallow, textual patterns [Gupta et al. 2017; Hindle et al. 2012; Pu et al. 2016].”
Repairing Bugs in Python Assignments Using Large Language Models (2022, background): “The ﬁeld of automated program repair (APR), which has a long history in the software engineering community [2]–[7], has introduced different approaches [8]–[11] to produce such automated repairs for student mistakes in introductory assignments.”
Learning and Evaluating Contextual Embedding of Source Code (2019, background): “…tasks, such as code completion (Hindle et al., 2012; Ray-chev et al., 2014), source code to pseudocode mapping (Oda et al., 2015), API-sequence prediction (Gu et al., 2016), program repair (Pu et al., 2016; Gupta et al., 2017), and natural language to code mapping (Iyer et al., 2018), among others.”
Automatic Grading and Feedback using Program Repair for Introductory Programming Courses (2017, background): “Recent works [3,14,19] use neural networks to “learn” the correct sequence of tokens from submissions that compile successfully, and use this knowledge to repair syntax errors in other sub-missions.”
COSET: A Benchmark for Evaluating Neural Program Embeddings (2019, unknown): “Early methods in the field applied NLP techniques to discover the textual patterns existed in the source code [5; 6; 7]; following approaches proposed to learn the syntactic program embedding from the Abstract Syntax Trees (AST) [8; 9; 10].”
DeepFix: Fixing Common C Language Errors by Deep Learning (2017, methodology): “Bhatia and Singh (2016) use prefixes implicated by a compiler and Pu et al. (2016) perform a brute force, enumerative search for localizing errors.”