Typilus: neural type hints

By Miltiadis Allamanis, Earl T. Barr, Soline Ducousso, Zheng Gao. 2020

150 citations recorded by the source corpus.

Type inference over partial contexts in dynamically typed languages is challenging. In this work, we present a graph neural network model that predicts types by probabilistically reasoning over a program’s structure, names, and patterns. The network uses deep similarity learning to learn a TypeSpace — a continuous relaxation of the discrete space of types — and how to embed the type properties of a symbol (i.e. identifier) into it. Importantly, our model can employ one-shot learning to predict an open vocabulary of types, including rare and user-defined ones. We realise our approach in Typilus for Python that combines the TypeSpace with an optional type checker. We show that Typilus accurately predicts types. Typilus confidently predicts types for 70% of all annotatable symbols; when it predicts a type, that type optionally type checks 95% of the time. Typilus can also find incorrect type annotations; two important and popular open source libraries, fairseq and allennlp, accepted our pull requests that fixed the annotation errors Typilus discovered.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Paleontology, Natural Language Processing, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Paleontology

In Natural Language Processing

In Program Analysis & Verification

Citations in the corpus

InCoder: A Generative Model for Code Infilling and Synthesis (2022, methodology): “Some recent work has developed supervised machine learning approaches for predicting type annotations for dynamically-typed languages including Python [69, 6, 50] and TypeScript [31, 66, 36].”
Contrastive Code Representation Learning (2020, background): “, 2020) and types (Pradel et al., 2019; Pandi et al., 2020; Wei et al., 2020; Allamanis et al., 2020; Bielik and Vechev, 2020; Allamanis et al., 2018) for various languages.”
Learning type annotation: is big data enough? (2021, methodology): “Typilus [4] also uses GNNs, but includes vector embeddings to allow an open type vocabulary (for Python).”
SimTyper: sound type inference for Ruby using type equality prediction (2021, methodology): “However, prior work [Allamanis et al. 2020; Hellendoorn et al. 2018; Malik et al. 2019], as well as our own dataset (§ 4), has found that the distribution of types in programs is Zipfian: a small number of types occur very frequently, while most types occur rarely.”
Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python (2021, methodology): “[64] PGM DeepTyper [26] 10K+ biRNN NL2Type [41] 1K LSTM TypeWriter [51] 1K HNN (3x RNNs) LAMBDANET [62] 100a GNN OptTyper [47] 100 LSTM Typilus [12] Unlimited GNN TypeBert [29] 40K BERT”
Learning to Predict User-Defined Types (2023, methodology): “Native types like number or string and library types like ArrayBuffer do not have declarations, and are very frequent [9], [12], [13], [15], [17], [18].”