Structural Language Models of Code

By Uri Alon, Roy Sadaka, Omer Levy, Eran Yahav. 2019

116 citations recorded by the source corpus.

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, methodology): “language models that generate code tokens [3, 36, 50, 62], rather than natural-language tokens, are the basis of intelligent IDEs [11] such as Deep TabNine [16] and Microsoft’s Visual Studio IntelliCode [46].”
InCoder: A Generative Model for Code Infilling and Synthesis (2022, background): “[7], who train an infilling-capable, AST-structured generative model of code on a smaller scale.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “n Wiley, Senior User Experience Designer for Google, said, “Google is no longer just ten blue links on a page, those days are long gone” [ReadWrite, 2010]. 3And indeed, programming languages as well [Alon et al., 2020, Feng et al., 2020]! 4https://www.blog.google/products/search/search-language-unde”
A systematic evaluation of large language models of code (2022, background): “Recently, LMs have shown impressive performance in modeling also source code, written in programming languages (Hindle et al., 2016; Hellendoorn & Devanbu, 2017; Alon et al., 2020; Karampatsis et al., 2020).”
GraphCodeBERT: Pre-training Code Representations with Data Flow (2020, background): “Neural Networks with Code Structure In recent years, some neural networks leveraging code structure such as AST have been proposed and achieved strong performance in code-related tasks like code completion (Li et al., 2017; Alon et al., 2019; Kim et al., 2020), code generation (Rabinovich et al.”
Integrating Tree Path in Transformer for Code Representation (2021, background): “Actually, lots of works have introduced these paths into many novel models [Kim et al., 2020, Alon et al., 2018, 2020].”
To What Extent do Deep Learning-based Code Recommenders Generate Predictions by Cloning Code from the Training Set? (2022, methodology): “[3] proposed a language-agnostic approach named Structural Language Model to tackle the problem of code completion.”
Code Prediction by Feeding Trees to Transformers (2020, background): “Among other flavors of code prediction are the ones where code after the prediction location, when available, is taken into account [9], [49], [51], [52], e.”
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code (2023, background): “Models of code have seen increasing accuracy in the past decade (Hindle et al., 2016; Raychev et al., 2014; Alon et al., 2020; Allamanis et al., 2018) for a variety of tasks such code completion (Fried et al.”
Learning Autocompletion from Real-World Datasets (2020, methodology): “[11] evaluated models on eleven large GitHub Java repositories.”
Modeling question asking using neural program generation (2019, background): “Some recent papers (Si, Yang, Dai, Naik, & Song, 2019; Alon, Sadaka, Levy, & Yahav, 2020) encode grammar with neural networks and use DRL to explicitly encourage the generation of semantically correct programs.”
Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models (2022, background): “[1] introduced structural language models that remove any restriction on the vocabulary or structure— the main limitation of program synthesis techniques.”
Neural Edit Completion (2020, methodology): “AST paths were shown to be an effective representation for predicting variable names, method names [Alon et al. 2019c], natural language descriptions [Alon et al. 2019a] and code completion [Alon et al. 2019b].”
Programming Puzzles (2021, methodology): “Move [i, j] means move top disk on tower i to j, with 1  i,j  3 def f2(moves: List[List[int]], num_disks=8): state = [1] * num_disks # All disks start at tower 1.”