Learning type annotation: is big data enough?

By Kevin Jesse, Prem Devanbu, Toufique Ahmed. 2021

43 citations recorded by the source corpus.

TypeScript is a widely used optionally-typed language where developers can adopt “pay as you go” typing: they can add types as desired, and benefit from static typing. The “type annotation tax” or manual effort required to annotate new or existing TypeScript can be reduced by a variety of automatic methods. Probabilistic machine-learning (ML) approaches work quite well. ML approaches use different inductive biases, ranging from simple token sequences to complex graphical neural network (GNN) models capturing syntax and semantic relations. More sophisticated inductive biases are hand-engineered to exploit the formal nature of software. Rather than deploying fancy inductive biases for code, can we just use “big data” to learn natural patterns relevant to typing? We find evidence suggesting that this is the case. We present TypeBert, demonstrating that even with simple token-sequence inductive bias used in BERT-style models and enough data, type-annotation performance of the most sophisticated models can be surpassed.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Program Analysis & Verification

Citations in the corpus

InCoder: A Generative Model for Code Infilling and Synthesis (2022, methodology): “Our filtering is similar to past work on generative models of code [18, 46, 68]: we remove files that contain any line longer than 3000 tokens or an average line length greater than 100 tokens, have less than 40% of their characters being alphanumeric or underscores, or appear to be automatically ge”
Do Machine Learning Models Produce TypeScript Types that Type Check? (2023, methodology): “Over the last few years, advances in model architectures and high-quality training data have led to type annotation prediction with high accuracy on individual type annotations [23, 24, 25, 38, 50].”
Multilingual training for Software Engineering (2021, background): “This approach is known to achieve state-of-the-art performance in both natural language processing, and software-related tasks [3, 4, 11, 18, 22, 32, 33, 35, 50, 75].”
Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python (2021, methodology): “Very recently, inspired by "Big Data", Jesse et al. [19] presented TypeBert, a pre-trained BERT model with simple token-sequence representation.”
Learning to Predict User-Defined Types (2023, methodology): “Advancements in neural networks are helpful for software engineering tasks, including type inference [9], [13], [14], [15].”