LambdaNet: Probabilistic Type Inference using Graph Neural Networks

By Jiayi Wei, Maruth Goyal, Greg Durrett, Işıl Dillig. 2020

119 citations recorded by the source corpus.

As gradual typing becomes increasingly popular in languages like Python and Typescript, there is a growing need to infer type annotations. While type annotations help with tasks like code completion and static error catching, these annotations cannot be fully inferred by compilers and are tedious to annotate by hand. This paper proposes a probabilistic type inference scheme for Typescript based on a graph neural network. Our approach first uses lightweight source code analysis to generate a program abstraction called a type dependency graph, which links type variables with logical constraints as well as name and usage information. Given this program abstraction, we then use a graph neural network to propagate information between related type variables and eventually make type predictions. Our neural architecture can predict both standard types, like number or string, as well as user-defined types that have not been encountered during training. Our experimental results show that our approach outperforms prior work in this space by 14% (absolute) on library types, while having the ability to make type predictions that are out of scope for existing techniques.

Subfields: Artificial Intelligence, Information Systems, Software, Natural Language Processing, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Software

In Natural Language Processing

In Graph Learning

Citations in the corpus

Program Synthesis with Large Language Models (2021, methodology): “Several methods have been proposed to do machine learning for type inference, for example, to add types to untyped code, such as when converting Javascript to Typescript [Hellendoorn et al., 2018, Pandi et al., 2020, Pradel et al., 2020, Wei et al., 2020].”
InCoder: A Generative Model for Code Infilling and Synthesis (2022, methodology): “Our filtering is similar to past work on generative models of code [18, 46, 68]: we remove files that contain any line longer than 3000 tokens or an average line length greater than 100 tokens, have less than 40% of their characters being alphanumeric or underscores, or appear to be automatically ge”
MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation (2023, methodology): “Code generation models have been applied to a variety of tasks, including test generation [18], docstring generation [19], code search [16], [20], type inference [21], [22], [23], and more [24].”
OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints (2020, methodology): “Probabilistic type inference has recently been proposed as an attempt to reduce the burden of writing and maintaining type annotations [Hellendoorn et al. 2018; Raychev et al. 2015; Wei et al. 2020].”
Learning to Predict User-Defined Types (2023, methodology): “The categories are as follows: Top 100 : The most frequent types such as native types int and string and types not considered user-deﬁned within the top 100 rank [12], [13], [17].”
Contrastive Code Representation Learning (2020, background): “, 2020) and types (Pradel et al., 2019; Pandi et al., 2020; Wei et al., 2020; Allamanis et al., 2020; Bielik and Vechev, 2020; Allamanis et al., 2018) for various languages.”
A Scalable and Extensible Approach to Benchmarking NL2Code for 18 Programming Languages (2022, background): “2020; Ahmed and Devanbu 2022), type inference (Wei et al. 2020; Hellendoorn et al. 2018; Pradel et al. 2020), and more (Drori et al.”
Learning type annotation: is big data enough? (2021, unknown): “Following LambdaNet[25] and OPTTyper, we only use the human annotated locations for evaluation but include the compiler-inferred types in training data.”
Learning semantic program embeddings with graph interval neural network (2020, methodology): “…introduction of graph neural networks to the programming domain [Allamanis et al. 2018; Li et al. 2016], they have been applied to a variety of PL tasks, such as program summarization [Fernandes et al. 2019], bug localization and fixing [Dinella et al. 2020], and type inference [Wei et al. 2020].”
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation (2022, methodology): “Code generation models have been applied to a variety of tasks, including test generation [19], docstring generation [20], code search [17, 21], type inference [22, 23, 24], and more [25].”
Solver-based gradual type migration (2021, methodology): “It is also possible to produce type migrations using supervised machine learning [Hellendoorn et al. 2018; Malik et al. 2019; Pradel et al. 2020; Wei et al. 2020].”
Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python (2021, methodology): “[64] PGM DeepTyper [26] 10K+ biRNN NL2Type [41] 1K LSTM TypeWriter [51] 1K HNN (3x RNNs) LAMBDANET [62] 100a GNN OptTyper [47] 100 LSTM Typilus [12] Unlimited GNN TypeBert [29] 40K BERT”
Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks (2020, methodology): “Prior work applying machine learning for program analysis also operates under these restrictions [2, 13, 21, 22, 28, 29].”
Migrating gradual types (2017, methodology): “Wei et al. (2020) developed LambdaNet for inferring types for TypeScript.”